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SECTION  I 

INTRODUCTION 


1.  IDENTIFICATION 

The  CAMP  Armonics  Benchmark  Suite  facilitates  the  evaluation  of  Ada  software  engineering  en¬ 
vironments  and  microprocessors  intended  for  use  in  armonics1  applications.  The  suite  features  both 
compilation  and  execution  benchmarks  to  measure  the  capabilities  of  compiler/run-time  systems.  All 
benchmarks  in  this  suite  are  portable  and  will  permit  comparisons  to  be  made  between  widely  different 
Ada  systems. 

This  volume  identifies  the  benchmarks  and  benchmark  drivers,  and  suggests  techniques  for  applying 
the  Benchmark  Suite.  In  addition,  the  structure,  purpose,  and  methodology  of  the  suite  are  explained  to 
familiarize  readers  with  the  suite  and  to  facilitate  the  evaluation  of  the  suite  by  engineers.  For  those 
interested  in  using  the  benclimarks,  a  guide  is  provided  in  Section  V.  Appendix  A  contains  data  collected 
in  the  process  of  running  the  Benchmark  Suite. 

2.  SYSTEM  OVERVIEW 

This  Armonics  Benclunark  Suite  serves  a  dual  purpose:  it  offers  a  means  for  assessing  the  perfor¬ 
mance  of  CAMP  parts  and,  at  the  same  time,  provides  support  for  evaluating  the  suitability  of  compiler 
systems  and  their  target  machines  to  armonics  applications. 

Ada  compiler  performance  is  tested  by  a  series  of  compilations,  based  on  CAMP  packages,  which 
require  a  compiler  to  process  complex  uses  of  Ada  generic  units.  These  advanced  (but  standard)  Ada 
features  are  used  heavily  in  the  CAMP  parts  and  are  central  to  the  development  and  use  of  reusable 
software. 

Other  benchmarks  of  the  suite  are  targeted  primarily  at  run-time  performance  issues  such  as  storage 
requirements,  execution  time,  and  computational  accuracy.  These  benchmarks  consist  of  a  selection  of 
CAMP  parts  which  have  been  chosen  as  representative  of  the  needs  of  armonics  applications.  Testing, 
using  these  benchmarks,  is  facilitated  by  embedding  (he  benchmarks  within  portable  drivers,  written  in 
Ada.  Effectively,  this  allows  the  benchmarks  to  run  themselves. 

The  Benchmark  Suite  can  support  a  number  of  benchmarking  scenarios: 

•  A  project  wishes  to  evaluate  compilers  for  use  in  the  development  of  a  reusable  parts  library.  The 
Tjhe  provides  test  code  for  evaluating  compiler/linker  systems. 

•  A  compiler  developer  wants  to  measure  the  performstnee  of  his  compiler/run-time  system  against  an 
established  rinndard.  A  group  of  benclimarks  documented  in  this  volume  provides  a  standard  for 
comparison  between  different  systems. 
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•  Ad  armonics  application  needs  data  on  the  memory  utilization  and  timing  efficiency  of  several 
compilers  in  order  to  select  an  appropriate  compiler  for  a  new  project.  The  benchmarks  provide 
opportunities  for  measuring  these  features  of  a  given  compiler. 

•  A  potential  user  of  CAMP  parts  wants  specific  performance  data  on  the  parts.  The  Benchmark 
Suite  gives  a  user  the  ability  to  measure  performance  for  a  selected  group  of  parts  on  varying 
architectures. 

•  A  scientific  application  requires  transcendental  functions  of  known  accuracy  on  a  specific  system 
and  is  considering  the  CAMP  polynomial  parts.  The  benchmarks  supply  data  on  the  scientific 
functions  of  the  CAMP  Polynomials  package. 

The  Benchmark  Suite  is  supplemented  by  a  set  of  procedures,  encoded  in  DEC  VAX  Digital  Com¬ 
mand  Language  (DCL).  By  performing  these  procedures  (or  their  equivalents  on  other  operating 
systems),  an  engineer  may  install,  compile,  and  run  the  various  benchmarks  efficiently.  Details  on  the  use 
of  the  Benchmark  Suite  and  its  command  procedure  environment  are  discussed  in  Section  V. 

3.  VOLUME  OVERVIEW 

This  report  contains  five  main  sections: 

1 .  Introduction:  Introduces  the  CAMP  Benchmark  Suite  and  this  volume. 

2.  Benchmark  Definitions:  Explains  the  system  of  classes  and  levels  by  which  the  benchmarks  are 
characterized.  This  section  also  introduces  key  terms  and  gives  a  tabular  summary  of  benchmarks 
contained  in  the  suite. 

3.  Purpose  and  Design:  Discusses  the  procedure  used  to  run  each  of  the  benchmarks  and  gives 
information  about  their  structure  and  scope.  For  each  benchmark,  this  section  provides  the  follow¬ 
ing  information  where  applicable: 

•  Benchmark  name  •  Benchmark  correct  outputs 

•  Compilation  structure  •  Data  to  be  recorded 

•  Benchmark  driver  design  •  Methods  for  recording  data 

•  Benchmark  inputs 

4.  Methodology:  Gives  the  overall  methodology  used  to  construct  the  Benchmark  Suite  in  terms  of 
portability,  validity,  and  usability. 

5.  Using  the  Benchmarks:  Explains  how  to  use  the  benchmarks  on  a  project.  Emphasis  is  placed  on 
making  use  of  the  suite  command  procedure  environment  to  facilitate  benchmarking. 
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SECTION  II 


BENCHMARK  DEFINITIONS 

1.  BENCHMARK  LEVELS 

The  CAMP  Armonics  Benchmark  Suite  supports  benchmarking  at  three  hierarchical  levels: 

•  TLCSC  benchmarks:  complete  operational  subsystems; 

•  LLCSC  benchmarks:  sequentially  driven  calls  to  integrated  CAMP  parts; 

•  Unit  benchmarks:  benchmarks  of  individual  parts.  (This  level  is  generally  reserved  for  the 
benchmarks  derived  from  CAMP  polynomial  parts  .) 

2.  BENCHMARK  CLASSES 

The  benchmark  suite  is  functionally  partitioned  into  three  classes.  The  compilation  benchmarks  test 
the  ability  of  an  Ada  compiler  to  process  source  code  typical  of  armonics  applications  and  reusable 
software.  Benchmarks  based  on  the  CAMP  Polynomials  scientific  function  package  are  called  the 
polynomial  benchmarks.  Finally,  the  benchmarks  developed  from  CAMP  higher-level  armonics  parts  are 
referred  to  as  integrated  execution  benchmarks.  The  following  subsections  define  the  three  classes  of 
benchmarks  in  greater  detail. 

a.  Compilation  Benchmark  Class 

The  compilation  benchmarks  test  an  Ada  compiler’s  ability  to  process  reusable  software.  The 
benchmarks  concentrate  on  the  complex  syntax  and  semantics  of  several  Ada  armonics-orieiited  im¬ 
plementations  using  CAMP  parts.  These  implementations  are  skeletal  in  that  they  do  not  actually  imple¬ 
ment  an  armonics  subsystem  but  merely  collect  the  necessary  CAMP  parts  via  generic  instantiation.  The 
instantiated  parts  are  invoked  in  the  benchmark  code  although  the  run-time  effects  of  the  invocations  are 
not  within  the  designed  scope  of  testing.  The  compilation  benchmarks  are  valid  tests  of  a  compiler  only 
up  to  (and  including)  the  linking  phase. 

b.  Polynomial  Benchmark  Class 

Tlie  Benchmark  Suite  includes  benclimarks  based  on  the  CAMP  Polynomial  parts  (part  number 
P688).  These  parts  cover  a  range  of  basic  mathematical  functions,  and  provide  a  variety  of  techniques  for 
obtaining  results.  For  each  benchmark,  the  benchmark  drivers  obtain  both  execution  time  data  and  func¬ 
tion  argument-result  pairs.  In  addition,  compilation  and  linkage  editing  of  the  polynomial  benchmarks 
afford  an  opportunity  to  collect  object  code  size  data  on  all  of  (lie  functions  of  the  Polynomials  package. 
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A  software  tool  provided  with  the  Benchmark  Suite  performs  accuracy  analysis  and  generates 
reports  for  the  polynomial  benchmarks.  This  tool  takes  the  output  produced  by  the  benchmarks  and  *■ 

generates  a  document  incorporating  time-consumption  data  and  function-result  accuracy  measurement. 

The  following  information  is  provided  by  the  tool: 

i. 

•  "Truth  values"  for  each  function  over  that  function's  benchmarked  domain: 

•  Absolute  error  in  the  result  of  each  argument-result  pair 

•  Relative  error  in  the  result  of  each  pair 

•  Maximum  relative  error  tracking  over  the  argument  domain 

•  Maximum  absolute  and  relative  error  over  the  argument  domain 

•  Root-mean-square  relative  error  over  the  argument  domain 

c.  Integrated  Execution  Benchmark  Class 

The  integrated  execution  benchmarks  test  aggregations  of  CAMP  armonics  parts.  These 
benchmarks  concentrate  on  three  of  the  major  operational  functions  supported  by  the  CAMP  parts: 

•  Waypoint  steering 

•  Navigation 

•  Kalman  filter 

In  the  waypoint  steering  and  navigation  cases  above,  data  is  gathered  on  CAMP  parts  in  the 
context  of  an  armonics  application.  This  method  has  the  virtue  of  testing  the  parts  in  the  kinds  of 
programs  in  which  they  will  actually  be  applied.  The  benchmark  based  on  the  CAMP  Kalman  filter  parts 
provides  data  on  these  parts  as  they  operate  in  a  unit  testing  environment.  This  method  permits  the  full 
inclusion  of  all  subprograms  in  the  CAMP  Kalman  filter  subsystem  TLCSCs. 

Output  data  from  the  integrated  execution  benchmark  drivers  consists  of  timing  and  result  data 
on  the  benchmark  subprograms.  The  timing  data  characterizes  the  execution  time  required  to  make  a 
single  call  to  the  benchmark  subprogram.  The  result  data  from  the  subprogram  may  be  compared  with  the 
standard  data  supplied  by  CAMP  as  part  of  the  Benchmark  Suite.  This  comparison  allows  the  engineer 
performing  the  benchmarks  to  spot  errors  and  inaccuracies  in  run-time  data  processing  on  his  system. 
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d.  Summary 

Table  1  summarizes  the  benchmarks  in  the  CAMP  Armonics  Benchmark  Suite.  For  each 
benchmark,  the  table  provides  the  following  information: 

•  Benchmark  name 

•  Benchmark  number:  a  unique  number  for  each  set  of  benchmarks,  corresponding  to  Section  Ill  of 
this  document.  This  number  gives  the  subsection  and  the  paragraph  of  Section  III  where  the 
benchmarks  are  described.  In  the  case  of  the  polynomial  benchmarks,  only  the  subsection  number  is 
applicable.  The  paragraph  number  tabulated  for  the  polynomial  benchmarks  is  only  for  serializa¬ 
tion. 

•  Level:  TLCSC  (T),  LLCSC  (L),  or  Unit  (U)  as  defined  above 

•  Class:  Compilation  (C),  Polynomial  (P),  or  Integrated  Execution  (I)  as  defined  above 

•  Objective:  the  objective  of  the  benchmark 

•  Description:  a  description  of  the  TLCSCs  used  in  the  benchmark  (for  the  polynomial  benchmarks, 
the  description  lists  the  polynomial  expansion  algorithm  tested) 

•  Data  to  be  recorded:  summary  of  data  values  generated  by  running  the  benchmark  and  recorded  in 
Appendix  A  of  this  report 
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TABLE  1.  CAMP  ARMONICS  BENCHMARK  SUMMARY 

(1  OF  2) 


Benchmark 

Name 

No. 

Lev. 

Ox. 

Objective 

Description 

Data 

to  Record 

Compilation  1 

2.1 

L 

C 

Test  compilability  of 
parts  needed  in  North 

Pointing  Navigation. 

j 

Packages  compiled: 

N_P_NavJ>arts, 

Polynomial_Parts. 

Gene  ral_Purpose_Math, 
Coord_Vector_Matrix_Alg, 
Stand  trd_Trig, 
Basic_Data_Type«, 
Converaion_Factors, 

WGS72  (Metric), 

WGS72  (Unit less), 
Univerial_Constants 

Object  code  size. 
Successful  compile. 
Compilation  time. 

Compilation  2 

2.2 

L 

C 

Text  compilability  of 
parts  needed  in  Waypoint 
Steering. 

Packages  compiled: 

W  aypoint_Sleering, 

Geometric_Operations, 

Coord_Vector_Matrix_Alg, 

Po!ynomial_Partx, 

General_Puipose_Math, 

Standard_T  rig, 

Basic_Data_Types. 

Conversion  Factors, 

WOS72  (Metric), 

Universal^ Constants 

Object  code  size. 
Successful  compile. 
Compilation  lime. 

Compilation  3 

2.3 

L 

C 

Test  compilability  of 
parts  needed  in  Kalman 

Filter. 

Packages  compiled: 
KalmFilterCompt  _H_Parts, 
KalmF  her_Common _Parts, 
PolynomisIParts, 

Gene  ral_Purpose_Math, 
Kahnan_Data_T ypea, 
C5er)eral_Vcctor_Matrix_Alg 

Object  code  size. 
Successful  compile. 
Compilation  lime. 

Integrated 
Execution  1 

3.1 

T 

« 

Test  execution  efficiency 
of  a  guidance  computation 
implementation. 

Packages  tested: 

W  ay  point_Sleering, 
Signal_Processmg 

Execution  time. 

Code  size. 

Result  data. 

Integrated 
Execution  2 

3.2 

T 

I 

Test  execution  efficiency 
of  a  navigation  operations 
implementation. 

Packages  rested: 

Comm_Navigation_Psrts, 

Direction_Cosine_Matrix 

OeneraI_Puipose_Math, 

Oeneral_Vector_Matrix_Alg. 

Wander_Az_Nav_Parls 

Execution  time. 

Code  size. 

Result  data. 

Integrated 
Execution  3 

3.3 

T 

I 

Test  execution  efficiency 
of  a  Kalman  Filter 
implementation. 

Packages  tesled: 
Abstracl_Data_Structs, 
Kabn_Filler_Common_Parts. 
Kalm_Ft  lie  r_Compt _U_Parts, 
Kalm_Filter_Compx_H_Parts, 

Execution  time. 

Code  size. 

Result  data. 

Sine 

Execution 

4.1 

U 

Test  execution  efficiency 
and  result  precision 
of  sine  function. 

Methods  tested  are: 

Taylor  Series, 

Modified  Taylor  Series, 
Hastings  Algorithm, 

Chehyshev  Polynomial, 

System  Functions 

Execution  time. 

Code  size. 

Result  Data 

1  Cosine 

Execution 

4-2 

U 

1 

Test  execution  efficiency 
and  result  precision 
of  cosine  function. 

Methods  tested  are: 

Taylor  Series, 

Modified  Taylor  Series, 
Hastings  Algorithm, 

1  lart  Algorithm, 

System  Functions 

Execution  time. 

Code  size. 

Result  Data 
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TABLE  1.  CAMP  ARMONICS  BENCHMARK  SUMMARY  (CONCLUDED) 


Bcnchmatk 

Name 

Tangent 

Execution 


Arcsine 

Execution 


Arccosine 

Execution 


Arctangent 

Execution 


Square  Root 
Execution 


Log  10 
Execution 


Log  N 
Execution 


Natural  Log 
Execution 


No.  Lev. 


4.10  U 


Objective 

Description 

Data 

to  Record 

Test  execution  efficiency 
!  and  result  precision 
of  tangent  function. 

i 

Methods  tested  are: 

Taylor  Series, 

Modified  Taylor  Series. 
Hastings  Algorithm, 

System  Function* 

Execution  time. 
Code  aize. 

Result  Data 

Test  execution  efficiency 
and  result  precision 
of  arcsine  function. 

Methods  tested  are: 

Taylor  Series, 

Fike  Semicircle, 

System  Functions 

Execution  time. 
Code  aize. 

Result  Data 

Test  execution  efficiency 
and  result  precision 
of  arccosine  function. 

Methods  tested  are: 

Taylor  Series, 

Fike  Semicircle, 

System  Function* 

Execution  time. 
Code  aize. 

Result  Data 

Test  execution  efficiency 
and  result  precision 
of  arctangent  function. 

Methods  tested  are: 

Taylor  Series, 

Continued  Fraction, 

Hastings  Algorithm, 

System  Functions 

Execution  time. 
Code  aize. 

Result  Data 

Test  execution  efficiency 
and  result  precision 
of  square  root  function. 

Methods  tested  are: 
Newton-Raphson 

Modified  Newton-Raphson 

Execution  time. 

Code  size. 

Result  Data 

Test  execution  efficiency 
and  result  precision 
of  log  10  function. 

Methods  tested  are: 

Taylor  Series, 

Cody-Wahe, 

System  Functiom 

Execution  time. 

Code  size. 

Result  Data 

Test  execution  efficiency 
and  result  precision 
of  log  n  function. 

Methods  tested  are: 

Taylor  Series, 

Cody-Waile, 

System  Functions 

Execution  time. 

Code  aize. 

Result  Data 

Test  execution  efficiency 
and  result  precision 
of  natural  log  function. 

Methods  tested  are: 

Taylor  Series. 

Cody-Wahe 

Execution  time. 

Code  size. 

Result  Data 

SECTION  III 

* 

PURPOSE  AND  DESIGN 

1.  GENERAL  REQUIREMENTS 

The  CAMP  Aimonics  Benchmark  Suite  meets  the  following  general  requirements: 

•  Utilizes  CAMP  parts  in  structures  which  simulate  their  actual  use  in  typical  user  applications 

•  Utilizes  test  data  modeled  on  typical  user  application  data 

•  Helps  assess  Ada  compilation  capabilities,  object  code  size,  execution  time,  and  output  results 

•  Permits  comparison  between  a  variety  of  host/target  combinations  using  different  Ada 
compiler/run-time  systems 

•  Allows  modification  to  meet  specific  needs  of  future  users 

•  Exhibits  high  portability 

•  Is  highly  automated 

a.  Identifying  Ada  Compiler  Inadequacies 

One  problem  faced  during  the  development  of  the  CAMP  parts  was  the  inability  of  some  Ada 
compilers  to  process  complex  generic  units.  This  is  important  because  Ada  generic  units  play  a  pivotal 
role  not  only  in  the  future  development  of  reusable  software,  but  also  in  the  application  of  that  software. 

In  order  to  identify  Ada  compiler  inadequacies  in  the  area  of  reusable  software  the  CAMP  benchmarks 
provide  Ada  source  code  benchmarks  which  heavily  utilize  Ada  generic  units. 

The  compilation  benchmarks  of  the  Armonics  Benchmark  suite  go  beyond  the  limited  scope  of 
testing  in  the  official  Ada  Compiler  Validation  Capability  (ACVC)  tests.  While  the  ACVC  tests 
demonstrate  conformance  to  the  Ada  language  specification,  (lie  effect  of  combining  language  features  in 
complex  ways  is  not  sufficiently  addressed.  The  CAMP  compilation  benchmarks  attempt  to  bridge  the 
gap  between  the  objectives  of  the  ACVC  tests  and  the  necessities  of  complex  software  applications.  It  is 
believed  at  this  point  that  very  few  ACVC-validated  Ada  compilers  will,  in  fact,  correctly  handle  the 
CAMP  compilation  benchmarks. 
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b.  Testing  Calculation  Accuracies 


The  CAMP  parls,  including  those  selected  as  benchmarks,  consist  of  portable  Ada  source  code. 
However,  certain  aspects  of  the  run-tiine  performance  of  the  parts  may  still  vary  from  system  to  system. 
Hie  accuracies  of  numeric  computations,  for  instance,  are  guaranteed  by  the  Ada  language  definition  to 
meet  the  minimum  requirements  specified  in  the  software,  but,  this  does  not  mean  that  different  compiler 
implementations  of  Ada  will  handle  numeric  computations  in  the  same  way.  A  compiler  is  free  both  to 
provide  wore  accuracy  than  is  requested  by  application  software,  and  to  support  less  accuracy  based  on 
the  limitations  of  the  target  machine.  For  this  reason,  the  results  of  calculations  performed  by  portable 
software  may  not  themselves  be  portable.  Differences  in  numeric  accuracies  and  range  limits  in  Ada 
systems  introduce  the  possibility  of  unanticipated  error  in  extensive  calculations.  This  factor  must  be 
considered  by  potential  users  of  the  CAMP  parts  as  it  would  have  to  be  by  users  of  any  software  (or 
hardware)  product. 

The  two  classes  of  execution  benchmarks  (polynomial  and  integrated  execution)  in  the  Ar¬ 
mories  Benchmark  Suite  address  the  issue  of  varying  computational  accuracies  in  different  Ada  systems. 
They  provide  a  standard  means  of  generating  data  from  the  kinds  of  complex  calculations  involved  in 
armories  applications. 

c.  Tesling  Time  and  Space  Performance 

An  important  performance  factor  in  real-time  embedded  (RTE)  environments  is  space  and  time 
efficiency:  Software  must  be  kept  small  because  hardware  must  be  kept  small  in  RTE  systems;  software 
must  also  operate  efficiently  because  of  the  tliroughput  requirements  of  real-time  processing.  The  execu¬ 
tion  benchmarks  of  the  Benchmark  Suite  support  execution-time  testing  of  CAMP  parts  as  they  operate 
on  various  Ada  compiler/target  machine  systems.  Selected  CAMP  parts  make  up  the  benchmarks  which 
cover  operations  common  to  many  armories  applications. 

The  size  of  the  object  code  generated  from  the  benchmarks  reflects  the  qualities  of  the  compiler, 
the  CAMP  parts,  and,  to  a  lesser  extent,  the  instruction  set  architecture  of  the  application  target  machine. 
Although  RTE  systems  are  being  built  with  more  and  more  memory,  hardware  capacity  and  its  associated 
costs  are  still  the  major  limiting  factor  in  increasing  the  computational  power  of  embedded  applications. 
The  execution  benchmarks  of  the  Benchmark  Suite  should  facilitate  the  evaluation  of  Ada  compiler/linker 
systems  based  on  object  code  size.  Linker  map  data,  obtained  by  compiling  and  linking  the  benchmarks, 
can  be  utilized  in  judging  an  Ada  system’s  appropriateness  to  an  embedded  application  in  the  light  of 
hardware  capacity  constraints. 
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2.  COMPILATION  BENCHMARKS 


The  purpose  of  the  compilation  benchmarks  is  to  determine  the  compilability  and  linkability  of  a 
large  selection  of  CAMP  parts  integrated  into  typical  armonics  application  groupings.  Results  from 
compiling  this  series  of  benchmarks  reflect  on  the  ability  of  Ada  compilers  to  correctly  process  CAMP 
parts.  Since  these  parts  are  both  reusable  and  armonics  application-oriented,  the  validity  of  the 
benchmarks  extends  strongly  to  these  two  areas. 

a.  Compilation  Croup  1 

CAMP  parts  utilized  as  benchmarks  in  Compilation  Group  1  represent  those  which  might  be 
needed  in  a  north-pointing  navigation  implementation.  The  structure,  components,  and  operating  proce¬ 
dure  of  this  compilation  benchmark  follow. 


USER  APPLICATION  PRCX3RAM 

pkg  V'HSqRt  b  naw  GPMatt.Squara_Root  ... 
pkg  AngVeISqRt  b  naw  GPMath.Squara_Roo* ... 
pkg  AcceISqRt  b  naw  GPMath.Squara_Root ... 
pkg  DtstSqRt  Is  naw  QpMath ,Squar*_Root ... 

pkg  VatVOpns  b  naw  CVMA.Vactor_Opns  ... 

pkg  AngVetVopns  b  naw  CVMA.Vackx_Opna  ... 
pkg  AccalVOpns  te  naw  CVMA.Vactor_Opns  ... 

pkg  DbtVOpns  b  naw  C VMA. V sctor_Opos ... 

fn  Cross Prod_AW_W  b  n«w  CVMA.Cross_Product  ... 

fn  CorAccel  b  rww  NPNav.Compute_Cork>Rs_AecebraHon 
pkg  RadOtCurv  b  naw  NPNav.Radtua_o(_Curvalura  ... 
pkg  Latlnt  Is  naw  NPNav.LatltudeJntegratlon  ... 


Figure  1.  Compilation  I  Structure 


•  Compilation  structure:  Figure  I  depicts  the  compilation  structure.  An  Ada  main  procedure  is 
compiled  in  the  context  of  several  CAMP  packages.  The  order  of  compilation  for  the  packages 
corresponds  to  the  partial  ordering  induced  by  the  context  clauses  (with  statements)  of  the  packages 
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and  driver  procedure.  A  command  file  in  (lie  tool  set  supplied  with  the  benchmark  suite  gives  a 
correct  compilation  order  and  compiles  the  compilation  benchmarks  automatically  on  VAX/VMS. 

•  Benchmark  driver  design: 

1.  Import  North_Pointing_Navigation_Parts  (CAMP  part  number  P001),  Genera l_Purpose_ 
Math  (P687),  Coordinate_Vector_Matrix_AIgebra  (P681),  Basic_Data_Types  (P621), 
WGS72_Ellipsoid_Metric_Data  (P611),  WGS72_Ellipsoid_Unilless_Data  (P613),  and 
SYSTEM. 

2.  Begin  main  procedure  definition. 

3.  Declare  types  and  subtypes  necessary  for  benchmark. 

4.  Instantiate  generic  units  t  ram  imported  packages. 

5.  Declare  objects  necessary  for  benchmark. 

6.  Invoke  instantiated  and  derived  subprograms  (executable  part  of  driver). 

7.  End  main  procedure  definition. 

•  Data  to  be  recorded: 

1 .  Successful  compilation; 

2.  Successful  link; 

3.  Object  code  size  (size  of  load  module  produced,  if  any); 

4.  CPU  time  consumed  by  the  compiler. 

•  Methods  for  recording  data:  The  source  files  for  this  compilation  benchmark  are  compiled  in  one 
group  with  the  source  files  for  the  others.  Error-free  compilation  is  indicated  by  the  compiler 
through  listings  or  by  some  other  mechanism.  CPU  time  consumption  is  noted  when  it  is  reported 
by  the  compiler.  The  driver  program  is  then  linked  and  the  size  of  the  executable  image  recorded. 


b.  Compilation  Croup  2 
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CAMP  parts  utilized  as  benchmarks  in  Compilation  Group  2  represent  those  which  might  be 
needed  in  the  waypoint  steering  of  a  missile  application.  The  structure,  components,  and  operating  proce¬ 
dure  of  this  compilation  benchmark  follow. 

•  Compilation  structure:  Figure  2  depicts  the  compilation  structure.  The  structure  is  similar  to  that 
of  Compilation  Group  1. 

•  Benchmark  driver  design: 

1.  Import  Waypoint_Steering  (CAMP  part  number  P661),  General_Purpose_Math  (P687), 
Coordinate_Vector_Matrix_Algebra  (P681),  Basic_Data_Types  (P621),  WGS72_Ellipsoid_ 
Metric_Data  (P61 1). 

2.  Begin  main  procedure  definition. 

3.  Declare  types  and  subtypes  necessary  for  benchmark. 

4.  Instantiate  generic  units  from  imported  packages. 

5.  Declare  objects  necessary  for  benchmark. 

6.  Invoke  instantiated  and  derived  subprograms. 

7.  End  main  procedure  definition. 

•  Data  to  be  recorded:  As  in  compilation  group  1 

•  Methods  for  recording  data:  As  in  compilation  group  1 

c.  Compilation  (I  roup  3 

CAMP  parts  utilized  as  benchmarks  in  Compilation  Group  3  represent  those  which  might  be 
needed  in  a  Kalman  filter  of  a  missile  application.  The  structure,  components,  and  operating  procedure  of 
this  compilation  benchmark  follow. 

•  Compilation  structure:  Figure  3  depicts  the  compilation  structure.  The  structure  is  similar  to  that 
of  the  other  two  compilation  groups. 

•  Benchmark  driver  design: 

1.  Import  KaIman_Filter_Complicatcd_H  (CAMP  part  number  P653)  and  Kalman_Filter_ 
Data_Types  (P622). 

2.  Begin  main  procedure  definition. 

3.  Declare  types  and  subtypes  necessary  for  benchmark. 

4.  Instantiate  generic  units  from  imported  packages. 
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5.  Invoke  instantiated  subprograms. 

6.  End  main  procedure  definition. 

•  Data  to  be  recorded:  As  in  the  other  two  compilation  groups 

•  Methods  for  recording  data-  As  in  the  other  two  compilation  groups 

3.  INTEGRATED  EXECUTION  BENCHMARKS 

This  section  describes  execution  benchmarks  based  on  CAMP  parts,  both  integrated  for  use  in  a 
typical  missile  application  and  in  a  unit-testing  environment.  The  purpose  of  the  integrated  execution 
benchmarks  is  to  generate  data  on  these  CAMP  parts  and  to  afford  an  opportunity  for  determining  code 
sizes. 
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a.  Integrated  Execution  I 

In  this  section  a  benchmark  based  on  a  guidance  computer  implementation  is  described.  Table 
2  lists  the  CAMP  parts  used  in  this  benchmark. 


TABLE  2.  CAMP  PART  BENCHMARKS  OF  INTEGRATED  EXECUTION  1 


TLCSC  NAME 

PART  NO. 

LLCSC  NAMES 

Waypoint  Steering 

P66I 

Compute  Turn  Angle  and  Direction 

Compute  Turning  and  Nontuming  Distances 
Distance  to  Current  Waypoint 

Steering  Vector  Operations  with  Arcsine 

Turn  Test  Operations 

Cross  Track  and  Heading  Error  Operations 

Signal  Processing  Parts 

P686 

Absolute  Limiter 

Upper  Lower  Limiter 

•  Benchmark  Driver  Design:  This  benchmark  is  based  on  the  guidance  computer  of  a  missile  ap¬ 
plication.  The  driver  consists  of  several  task  bodies  declared  in  the  declaration  section  of  a  main 
procedure.  These  tasks  are  activated  after  the  elaboration  of  the  driver  declaration  section.  A  null 
executable  part  of  the  driver  runs  to  completion  and  awaits  the  termination  of  the  tasks. 

The  tasks  call  the  benchmark  subprograms  in  the  course  of  execution.  A  counter  keeps  track  of 
calls  to  a  central  message  management  task.  When  the  counter  value  reaches  a  certain  level,  the 
task  is  aborted  and  becomes  abnormal.  As  the  other  tasks  attempt  to  rendezvous  with  the  aborted 
task,  they  are  forced  to  select  a  "terminate"  entry.  Then,  these  tasks  also  become  abnormal.  When 
the  child  tasks  of  the  driver  have  all  become  abnormal,  the  driver  terminates  execution. 

*  Data  to  be  recorded: 

•  Execution  time 
-  Code  sizes 
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Result  data 


•  Methods  for  recording  data:  Execution  time  is  obtained  directly  from  the  benchmark  driver.  The 
code  sizes  of  the  various  CAMP  parts  may  be  taken  from  linker  map  files.  Result  data  is  also 
generated  directly  by  the  benchmark. 

•  Benchmark  inputs:  Before  execution,  the  benchmark  driver  requests  data  about  the  system:  the 
compiler  used,  the  compiler  host,  and  the  compiler  target.  Then  iteration  values  are  requested  to 
tell  the  driver  how  many  times  to  execute  a  benchmark  subprogram.  The  benchmarks  themselves 
are  supplied  with  hard-coded  input  data  by  the  driver  software.  These  inputs  are  coded  as  variables 
to  preserve  the  functionality  of  the  benchmarks,  which  would  not  normally  process  static  data. 

•  Benchmark  correct  outputs:  A  file  containing  standard  output  is  supplied  with  the  benchmark  suite. 
It  should  be  used  for  comparison  with  the  actual  benchmark  output. 


b.  Integrated  Execution  2 

In  this  section  a  benchmark  based  on  a  navigation  computer  implementation  is  described.  Table 
3  lists  the  CAMP  parts  used  as  benclunarks. 

TABLE  3.  CAMP  PART  BENCHMARKS  OF  INTEGRATED  EXECUTION  2 


TLCSC  NAME 

PART  NO. 

LLCSC  NAMES 

Common  Navigation  Pafli 

POOI 

Update  Velocity 

Compute  Ground  Velocity 

Compute  Gravitational  Acceleration  Sin  Lat  In 

Wander  Azimuth  Navigation  Part* 

P002 

Radius  of  Curvature 

Compute  East  Velocity 

Compute  North  Velocity 

Compute  Latitude  using  2-Value  Arclan 

Compute  Longitude  using  2- Value  Arc  tan 

Compute  Wander  Azimuth  Angle 

Earth  Rotation  Rale 

Earth  Relative  Navigation  Rotation  Rates 

Compute  Coriolis  Acceleration 

Total  Platform  Rotation  Rate 

Direction  Cosine  Matrix 

P644 

CNE  Operations 

General  Vector  Matrix  Algebra 

P682 

Matrix  Matrix  Multiply  Restricted 

General  Purpose  Math  Parts 

P687 

Accumulator 

•  Benchmark  Driver  Design:  This  set  of  three  benchmark  drivers  is  based  on  the  navigation  opera¬ 
tions  of  a  missile  application.  Each  driver  uses  the  same  basic  Ada  linkage  closure  of  units,  sub¬ 
stituting  dummy  code  as  appropriate.  The  first  phase  of  the  benchmarking  run  is  done  by  the 
driver,  "Execute_Navigator_Test,"  which  calls  most  of  the  benchmark  subprograms.  The  remain¬ 
ing  benchmark  subprograms  are  called  by  two  drivers  embedded  in  the  executable  part  of  the 
Navigation  Operations  package. 

•  Data  to  be  recorded: 
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1.  Execution  time 


t 

2.  Code  size 

3.  Result  data 

•  Methods  for  recording  data:  As  in  Integrated  Execution  I 

•  Benchmark  inputs:  As  in  Integrated  Execution  1 

•  Benchmark  correct  outputs:  As  in  Integrated  Execution  1 


c.  Integrated  Execution  3 

In  this  subsection  a  benchmark  based  on  the  CAMP  Kalman  filter  unit  tests  is  described.  Table 
4  lists  the  CAMP  parts  used  as  benchmarks. 

TABLE  4.  CAMP  PART  BENCHMARKS  OF  INTEGRATED  EXECUTION  3 


TLCSC  NAME 

PART  NO. 

LI. CSC  NAMES 

Kalman  Filler  Common  Parti 

P65I 

Error  Covariance  Matrix  Manager 

State  Transition  and  Process  Noise  Matrices  Manager 

State  Transition  Matrix  Manager 

Kalman  Filler  Compact  H  Parts 

P652 

Compute  Kalman  Gaina 

Update  Error  Covariance  Matrix 

Update  Stale  Vector 

Sequentially  Update  Covariance  Matrix  And  Stale  Vector 
Kalman  Update 

Update  Error  Covariance  Matrix  General  Form 

Kalman  Filler  Complicated  H  Parts 

P653 

Compute  Kalman  Gam 

Update  Error  Covariance  Matrix 

Update  State  Vector 

Sequentially  Update  Covariance  Matrix  And  Stale  Vector 
Kalman  Update 

Update  Error  Covariance  Matrix  General  Form 

•  Benchmark  driver  design:  The  three  drivers  of  this  benchmark  are  based  on  the  unit  tests  of  the 
CAMP  Kalman  filler  parts  (P651,  P652,  and  P653).  Three  main  procedures  import  the  three  Kal¬ 
man  TLCSCs  and  call  the  benchmark  subprograms  within  them. 

•  Data  to  be  recorded: 

-  Execution  time 

-  Code  size 

•  Result  data 

•  Methods  for  recording  data:  As  in  the  other  two  integrated  execution  benchmarks 

•  Benchmark  inputs:  As  in  the  other  two  integrated  execution  benchmarks 

•  Benchmark  correct  outputs:  As  in  the  other  two  integrated  execution  benchmarks 
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4.  POLYNOMIAL  BENCHMARKS 

Tlic  purpose  of  the  polynomial  benchmarks  is  to  generate  run-time  data  on  the  "slide  rule"  functions 
of  the  CAMP  Polynomials  package  and  to  provide  an  opportunity  for  collecting  object  code  size  data. 
The  run-time  data  on  the  benchmarks  is  produced  by  benchmark  drivers  and  includes  information  both  on 
the  time-consumption  of  the  benchmarks  and  the  numeric  output  they  produce. 

Table  5  presents  a  summary  of  the  execution  benchmarks  which  have  been  created  from  the  CAMP 
Polynomials  parts.  Entries  marked  "X"  indicate  a  function  and  a  numerical  algorithm.  For  each  math¬ 
ematical  function  of  the  CAMP  Polynomials  package,  all  of  the  available  algorithm  implementations  are 
used  as  benchmarks.  The  floating  point  types  of  the  arguments  and  results  are  varied  according  to  the 
number  of  terms  in  each  algorithm’s  polynomial  expansion.  For  example,  an  algorithm  for  a  5-term 
polynomial  expansion  may  be  instantiated  to  use  6  floating-point  digits  while  an  algorithm  for  a  7-term 
expansion  is  instantiated  to  use  9  digits. 


TABLE  5.  CAMP  POLYNOMIAL  PARTS  EXECUTION  BENCHMARKS 


Function 

Taylor 

Series 

Modified 

Taylor 

Scries 

Hastings 

Cltebyshev 

System 

Functions 

Halt 

Fifce 

Continued 

Fraction 

Newton- 

Raptwon 

Modified 

Newton- 

Raphaon 

Cody- 

Wade 

Sine 

X 

X 

X 

X 

X 

Cosine 

X 

X 

X 

E  ■ 

Tangent 

X 

X 

X 

X 

1 

I 

X 

Arcsine 

X 

mm 

.  ■ 

gpi 

Arccosine 

X 

Efl 

Arctangent 

X 

X 

S3 

E 

■  S 

X 

Square  root 

mk 

X 

X 

Log  10 

X 

mm 

■M 

El 

Log  2 

X 

1 

Nat.  Log 

■ 

I 

B 

B 

I 

B  B 

Tables  6  through  12  present  details  of  the  polynomial  function  benchmarking.  An  "X"  entry  in  a 
table  indicates  an  algorithm  for  computing  a  function  and  the  number  of  terms  of  that  algorithm  to  be 
applied  in  the  computation.  Detail  tables  are  not  included  for  the  various  log  functions  shown  in  Table  5 
since  the  log  function  testing  is  coufined  to  the  Polynomials  Cody-Waite  LLCSC.  Term  counts  are  not 
applicable  to  the  parts  of  this  LLCSC. 

•  Benchmark  driver  design  (for  all  polynomial  benchmarks): 

1 .  Import  the  CAMP  Polynomials  package  (P688).  Benchmarking_Tools  package,  and  the 
Polynomial_Benchmark  package. 

2.  Define  a  floating  point  type  of  some  precision. 

3.  Instantiate  a  Polynomials  package  LLCSC  for  the  defined  type. 

4.  Instantiate  the  Polynomial_Benchmark  package  for  the  defined  type. 
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TABLE  6.  DETAIL  OF  SINE  PERFORMANCE  BENCHMARKS 


Number 

of 

Terms 

Tsylor 

Series 

Modified 

T.yior 

Scries 

IlMfinga 

AJgivitltfn 

Chebyshev  System 

Potyaomiel  Fs  notion* 

4 

X 

X 

X 

J 

X 

X 

X 

X 

6 

X 

X 

7 

X 

X 

S 

X 

X 

VAX 

X 

5.  Instantiate  the  Benchmark  procedure  (procedure  named  Benchmark)  from  the  Polynomial. 
Benchmark  package.  Pass  in  a  function  subprogram  as  a  generic  actual  parameter.  This  is 
the  subprogram  to  be  benchmarked.  Pass  in  an  identity  function  from  the  Benchmarking. 
Tools  package  as  another  generic  actual  parameter.  This  subprogram  helps  to  compensate 
for  time  costs  associated  with  the  design  of  the  benchmark  driver  software.  A  new 
Benchmark  procedure  instantiation  is  required  for  each  subprogram  benchmark  from  the 
Polynomials  package. 

6.  Request  the  system  information.  This  includes  the  name  of  the  compiler  used  to  compile 
the  benchmark  and  the  names  of  the  host  and  target  machines  of  the  compiler.  This  data  is 
incorporated  into  the  benchmark  driver  output  to  note  the  environment  in  which  the 
benchmark  is  being  carried  out.  See  "Benchmark  Correct  Outputs"  below. 

7.  Request  the  number  of  iterations  to  use  for  each  benchmark.  Separate  numbers  are  re¬ 
quested:  one  for  the  number  of  iterations  to  use  when  timing  the  benchmark,  the  other  for 
the  iterations  to  use  when  collecting  data  from  the  benchmark. 

8.  Call  the  instantiated  Benchmark  procedures.  These  procedures  time  the  benchmark  sub¬ 
program  over  a  selected  domain.  They  also  provide  input  and  output  data  echoing  for  the 
benchmark  subprogram  over  the  argument  domain. 

9.  End  of  benchmark  definition. 

*  Benchmark  inputs:  System  information  and  iteration  values  are  supplied  at  run-time  via  the  con¬ 
sole. 
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•  Benchmark  correct  outputs:  The  benchmarks  produce  time-consumption  data  as  well  as:  echoed 

t  system  information  (noted  above),  an  Ada  enumeration  literal  for  the  function  being  benchmarked 

(e.g.  SINEJR  for  radian  sine),  and  ordered  pairs  of  benchmark  subprogram  input  and  output.  Ac¬ 
curacy  of  the  subprogram  output  is  determined  by  an  analysis  program  supplied  with  the 
Benchmark  Suite.  This  program  uses  the  VAX  Ada  Math  library  (MATH_L1B)  to  obtain  truth 
values.  Absolute  error  in  a  benchmark  subprogram  is  calculated  as  the  difference  between  the  result 
of  that  subprogram  and  the  truth  value  result  for  a  given  argument.2 

•  Data  to  be  recorded: 

1 .  Execution  time  for  one  call  to  each  Ada  subprogram  benchmark 

2.  Code  size 

3.  Arguments  and  benchmark  function  results  for  those  arguments 

4.  System  information  collected  at  the  beginning  of  the  run 

•  Methods  for  recording  data:  Time-consumption  data  is  recorded  and  reported  automatically  by  the 
benchmark  drivers.  Input  data,  output  data,  system  information  echoing,  and  an  enumeration  literal 
representing  the  kind  of  function  benchmarked  are  also  reported  automatically.  Analyzed  output  is 
obtained  by  passing  the  benchmark  driver  output  through  the  analysis  program  Analyze.  Code  size 
information  is  retrieved  from  linker  maps. 


TABLE  7.  DETAIL  OF  COSINE  PERFORMANCE  BENCHMARKS 


Number 

Modified 

of 

Taylor 

Taylor 

Hasling* 

Hart 

System 

Terms 

Scries 

Series 

Algorithm 

Algorithm 

F»  notions 

4 

X 

X 

X 

5 

X 

X 

X 

X 

6 

X 

X 

7 

X 

X 

S 

X 

X 

VAX 

X 

i» 


$ 

2No(e:  a  small  amount  of  error  in  induced  by  conversion  to  and  from  text  representations  of  floating-point  numbers. 
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TABLE  8.  DETAIL  OF  TANGENT  PERFORMANCE  BENCHMARKS 


TABLE  9.  DETAIL  OF  ARCSINE  PERFORMANCE  BENCHMARKS 


TABLE  10.  DETAIL  OF  ARCCOSINE  PERFORMANCE  BENCHMARKS 
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TABLE  12.  DETAIL  OF  SQUARE  ROOT  PERFORMANCE 

BENCHMARKING 


2i 


SECTION  IV 
METHODOLOGY 

The  following  paragraphs  explain  methods  used  in  constructing  the  Armonics  Benchmark  Suite. 
These  sections  discuss  the  overall  design  aspects  of  the  suite  as  applied  to  the  problems  of  portability, 
validity,  and  automation  of  data  collection. 

I.  PORTABILITY 

Like  the  CAMP  parts  in  general,  the  Benchmark  Suite  is  highly  portable,  extending  its  usability  and 
repeatability  to  many  different  Ada  systems.  The  CAMP  parts  selected  as  benchmarks  use  only  Mil- 
Std-1815A  Ada  code,  as  do  the  drivers  which  automate  much  of  the  benchmarking.  Whenever  optional 
Ada  features  are  applied  (e.g.,  pragma  PAGE),  their  effects  are  irrelevant  and  they  may  be  freely  ignored 
by  Ada  compilers. 

Input  to  and  output  from  the  benchmarks  is  limited  to  the  use  of  the  console,  obviating  file  I/O 
implementation  in  the  target  system.  While  a  filing  system  is  desirable  in  order  to  retain  output,  the 
console  I/O  approach  possesses  greater  versatility  since  many  embedded  computers  (and  hosted 
debugger/simulators  for  the  same)  may  not  fully  support  file  I/O.  In  such  cases,  the  use  of  file  I/O  could 
make  the  benchmarks  difficult  to  transport  to  the  kinds  of  architectures  for  which  they  are  intended. 
Moreover,  the  use  of  console  I/O  does  little  to  impede  the  retention  of  benchmark  data  on  a  filing  system. 
The  Ada  language  and  most  operating  systems  supply  trivial  mechanisms  for  redirecting  console  output 
to  files. 

The  tool  set  which  accompanies  the  benchmark  suite  is  system-dependent  and,  as  previously  noted, 
consists  of  VAX  DCL  command  procedures  and  some  non-portable  Ada.  Designed  to  automate  the  com¬ 
pilation  and  execution  of  the  benchmarks,  this  tool  set  supports  two  possible  uses:  For  VAX/VMS  users, 
the  tool  set  substantially  automates  benchmarking;  for  users  of  other  systems,  the  tools  are  well 
documented  to  permit  a  knowledgeable  user  to  modify  them  or  use  them  as  a  guide  for  performing  the 
benchmarks  on  his  own  system.  A  more  detailed  treatment  of  the  tools  is  presented  in  Section  V. 

In  order  to  automate  the  timing  of  benchmark  executions  in  a  portable  way,  the  benchmark  drivers 
use  facilities  from  the  Ada  CALENDAR  package.  Although  differences  in  the  implementation  of  this 
package  may  exist  between  systems,  these  differences  are  minor  enough  that  their  effects  can  be  min¬ 
imized.  The  design  of  the  benchmark  drivers  attempts  to  lake  advantage  of  similarities  in  Ada  systems 
supporting  the  CALENDAR  package,  while  accounting  for  the  differences  that  exist. 
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For  example,  (be  duration  of  a  basic  clock  cycle  (Ada  SYSTEM.T1CK)  may  vary  from  system  to 
»  system  and  may  be  quite  large  with  respect  to  the  benchmark  execution  times.  This  requires  a  benchmark 

driver  to  execute  its  benchmark  many  times  in  order  to  arrive  at  a  reasonable  one-call  execution-time 
estimate.  By  examining  the  Ada  constant  SYSTEM.TICK,  the  drivers  are  able  to  calculate  the  number  of 
*  benchmark  executions  necessary  to  arrive  at  a  set  timing  accuracy.  Conversely,  given  the  number  of 

executions  used  in  benchmarking,  the  drivers  calculate  an  estimated  accuracy  on  the  time-consumption 
data  obtained  from  those  executions. 

2.  COMPILATION  BENCHMARK  METHODS 

The  compilation  benchmarks  are  valid  tests  of  a  compiler  but  do  not  apply  to  code  generation.  They 
are  intended  to  force  an  Ada  compiler/linker  system  to  fail  when  it  contains  errors  in  semantic  analysis,  in 
library  management,  or  in  linkage  editing. 

To  pass  the  compilation  benchmark  test,  a  compiler  must  process  the  associated  Ada  source  code 
without  signaling  any  errors  or  ending  abnormally.  Limited  warnings  are  allowed  since  the  Ada  language 
allows  a  compiler  some  flexibility.  For  example,  a  compiler  can  warn  that  it  has  made  an  optimization  or 
ignored  an  optional  pragma.  Warnings  about  program  semantics,  however,  should  not  be  generated,  nor 
should  die  compiler  or  linker  encounter  fatal  errors  in  library  management  or  load  module  generation. 

The  compilation  benchmarks  are  performed  by  a  DCL  command  procedure.  This  procedure  must  be 
supplied  with  a  compiler  invocation  command  and  a  linker  invocation  command.  It  then  proceeds  to 
apply  these  commands  to  the  necessary  Ada  source  files  in  a  correct  order.  The  procedure  can  be  altered 
or  used  as  a  guide  when  benchmarking  on  systems  other  than  VAX/VMS. 

The  compilation  benchmarks  were  validated  by  successfully  compiling,  linking,  and  running  their 
source  code  on  a  highly  reliable  Ada  corapiler/linker  system.  The  system  on  which  the  validation  was 
performed  is  ACVC  validated  and  produced  no  error  messages  or  warnings  in  the  course  of  compiling, 
linking,  and  running  the  compilation  benchmark  source  code. 

3.  EXECUTION  BENCHMARK  METHODS 

a.  Collecting  Valid  Timing  Data 

Drivers  of  the  execution  benchmarks  collect  time-consumption  data  through  the  use  of  the  Ada 
CALENDAR  package.  As  noted  above,  the  Ada  constant  SYSTEM.TICK  varies  between  systems  and  is 
usually  quite  large.  Because  of  this  the  benchmarks  are  called  repeatedly  for  the  sake  of  timing  accuracy. 
The  number  of  repetitions  necessary  to  achieve  microsecond  accuracy  is  computed  relative  to 
SYSTEM.TICK  and  reported  by  the  benchmark  drivers  at  run-time.  This  enables  a  user  of  the 
Benchmark  Suite  to  decide  on  the  number  of  repetitions  to  actually  use  in  benchmarking. 

The  computed  number  of  repetitions  is  not  used  automatically  since  the  resulting  processing 
time  of  the  benchmark  drivers  might,  in  some  cases,  become  prohibitively  long.  Large  numbers  of 
repetitions  on  slow  systems  may  consume  a  substantial  amount  of  CPU  time.  Reducing  the  number  of 
repetitions  proportionally  reduces  both  the  driver  CPU-time  expense  and,  unfortunately,  the  accuracy  of 
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the  collected  benchmark  tunings.  The  engineer  who  runs  the  benchmarks  must,  therefore,  make  a  trade* 
off  with  respect  to  timing  accuracy  and  the  use  (or  overuse)  of  computational  resources. 

In  order  to  ensure  valid  timings  of  the  benchmarks,  a  number  of  precautions  are  built  into  the 
system.  Code  optimizations  which  might  affect  the  integrity  of  tlie  time-consumption  data  are  selectively 
defeated  while  other  optimizations  remain  untouclied.  The  methods  used  are  similar  to  those  used  in 
ACVC  tests  to  prevent  the  compile  time  removal  of  code  which  is  being  tested.  These  methods  entail  the 
use  of  identity  functions  and  tautological  BOOLEAN  functions  to  deprive  the  compiler  of  optimization 
opportunities.  Here,  the  goal  of  optimization  suppression  is  essentially  to  "fool"  the  compiler  by  reducing 
its  available  control-flow  information  at  compile  time. 

For  example,  consider  Figure  4.  The  identity  function  used  in  the  first  part  of  this  Ada  fragment 
prevents  a  compiler  from  propagating  the  constant  H5"  into  the  timing  loop  in  place  of  the  variable 
Argument.  If  this  propagation  were  allowed  to  occur,  tlie  measured  lime  for  the  subprogram  call  could  be 
reduced.  The  reduction,  however  would  be  due  to  the  static  nature  of  the  argument,  a  circumstance  which 
would  not  often  occur  if  the  subprogram  were  used  in  an  application.  This  method  of  optimization 
suppression  is  used  in  the  integrated  execution  benchmarks  where  constants  are  often  supplied  as  ar¬ 
guments. 


Figure  4.  Identity  Function  Defeats  Constant  Propagation 


In  a  second  example.  Figure  5  shows  the  use  of  a  tautological  BOOLEAN  function  to  prevent 
the  removal  of  a  "dead"  assignment.  The  function,  Snow_Is_While,  always  returns  the  value  TRUE, 
although  this  is  not  known  at  compile  time  by  the  compiler  (the  body  of  the  function  is  separate).  Since 
the  flow  of  control  is  not  known,  the  compiler  cannot  remove  the  assignment  to  the  variable  Gross_Time. 

If  this  optimization  were  allowed,  the  compiler  could  move  the  evaluation  of  the  Gel_Elapsed_Time_ 

Since_Start  function  into  the  expression  assigned  to  the  variable  Next_Time_Used.  While  such  a  move 
would  not  alter  the  logical  meaning  of  the  program,  a  small  effect  on  the  time  data  would  result.  This 
optimization  suppression  is  used  in  the  polynomial  benchmarks. 

It  should  be  noted  that  these  techniques  do  not  have  the  negative  effect  of  inhibiting  desired 
optimizations.  An  Ada  compiler  is  free  to  optimize  all  unprotected  source  code,  including  the  code  • 

bodies  of  the  benchmarks  themselves.  It  should  also  be  noted,  however,  that  these  techniques  are  not 
fool-proof.  Conceivably,  a  smart  enough  compilcr/Jinker  could  outwit  the  optimization  suppressions 
described  here.  « 


24 


—  loop  d*t*niMi  onchMd 


•tart_Tij*or; 

foe  Xndax  in  leatJUn;*  loop 

toywit  :•  Identity  (Irpnont) ; 
and  loop; 

If  faow_Xa_miito  than  —  baltncai  ’if  balow . 

Omkatd  flaw  :■  Sot  Blapaod  Tina  Slnoa  Start; 
and  If; 

•tart_Tl»ar; 

for  Xadax  In  loaaJUn^a  loop 

Irpaut  :■  Identity  (Aronant) ; 

Kaaalt  ;a  Banchaark_ruaotlon  (Argument) ; 
and  loop; 

If  S»ow_I«_Whlta  than 

Oroaa_Tlna  :■  Sat_Elapaad  Zl>a_Slnca_Start; 
and  If; 

Mat  Tina  Used  :■  Sroaa  Xlaa  -  Overhead  Time; 


Figure  5.  Tautological  Function  Prevents  Assignment  Removal 


Figure  5  also  illustrates  the  collection  of  time-overhead  to  calibrate  benchmark  timings.  Time 
overheads  are  calculated  at  run-time  and  are  used  to  offset  the  effects  of  the  timing  method  and 
benchmark  code  idiosyncracies.  In  the  case  of  the  polynomial  benchmarks,  the  overheads  are  often 
negligible,  since  none  of  these  benchmarks  require  initialization. 

On  the  other  hand,  the  execution  time  overheads  of  the  integrated  execution  benchmarks  are 
usually  appreciable  due  to  the  parameter  requirements  of  the  benchmark  subprograms.  Many  of  the 
higher-order  CAMP  parts  used  as  integrated  execution  benchmarks  have  side  effects  and  in-out 
parameters.  Each  execution  of  such  a  benchmark,  therefore,  has  a  cumulative  effect  which  may  produce 
an  exception  after  many  iterations.  In  order  to  counteract  this  effect,  many  of  the  integrated  execution 
benchmarks  must  be  re-initialized  prior  to  each  call,  a  process  which  adds  very  significantly  to  overhead. 
The  resolution  of  this  problem  is  transparent  to  the  user  and  is  accomplished  by,  as  in  the  polynomial 
benchmarks,  implementing  automatic  overhead  correction  in  the  benchmark  drivers. 

Despite  all  of  the  precautions  taken  to  ensure  the  validity  of  the  time-consumption  data,  in¬ 
accuracies  may  still  occur.  The  benchmark  drivers  may  overestimate  benchmark  execution  times  when 
asynchronous  events  take  place  in  the  midst  of  timing.  For  example,  if  the  benchmark  drivers  are 
operated  on  a  time-sharing  operating  system,  they  will  compete  with  other  processes.  Since  the  CALEN¬ 
DAR  package  operates  on  wall-clock  time  rather  Ilian  CPU  time  consumption,  the  benchmarks  will  ap¬ 
pear  to  execute  longer  as  their  CPU  time  fraction  is  reduced. 


—  function  always  TKUB 

—  ranonl  prmnttd 
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Problems  of  ibis  kind  are  beyond  the  control  of  the  benchmark  drivers.  The  effects  of 
asynchronous  events  on  the  benchmark  timings  may  be  minimized,  but  inaccuracies  should  nevertheless  * 

be  assumed.  When  asynchronous  interference  with  the  benchmarks  is  relatively  uniform,  the  benchmark 
execution  times  wil*  lengthen  proportionally  to  their  synchronous  execution  times.  Benchmarks  which,  by 
themselves,  take  a  relatively  long  time  to  run  will,  of  course,  show  a  relatively  larger  dilation  in  measured 
execution  time.  While  this  effect  may  be  undesirable,  it  can  usually  be  taken  into  account. 

Moreover,  timings  which  include  asynchronous  interference,  typical  of  an  operating  environ¬ 
ment,  are  quite  valid.  In  such  an  environment,  estimates  based  on  the  CPU  time  consumption  alone 
would  be  unrealistically  low.  The  true  throughput  of  an  application  is  a  function  of  both  the  application 
execution  speed  and  the  typical  amount  of  asynchronous  interference  with  which  the  application  must 
contend. 

b.  Collecting  Benchmark  Output  Data 

In  addition  to  timing  data,  the  polynomial  benchmarks  provide  data  for  use  in  determining  the 
accuracy  of  Polynomials  (CAMP  package)  function  results.  For  each  function  benchmarked,  both  input 
and  output  are  reported  at  equal  intervals  over  a  selected  argument  domain.  This  permits  the  result  ac¬ 
curacies  of  the  functions  to  be  checked  against  appropriate  truth  values.  For  users  with  access  to  VAX 
Ada,  accuracy  analysis  and  report  generation  can  be  accomplished  automatically  using  a  tool  provided 
with  the  Benchmark  Suite.  Other  users  may  make  use  of  this  tool  by  modifying  it  as  explained  in  section 
V.  It  should  be  noted,  however,  that  the  accuracy  analysis  tool  is  not  required  in  order  to  run  the 
benchmarks. 

Armonics  subsystem  output  data  is  also  produced  by  the  integrated  execution  benchmarks  al¬ 
though  automatic  checking  of  this  data  is  not  supported.  Most  of  the  output  from  these  benchmarks  is 
produced  in  an  ad  hoc  format  which  does  not  lend  itself  to  automatic  analysis.  Nevertheless,  the  correct¬ 
ness  of  the  data  may  be  checked  by  manual  comparison  with  standard  output  files  supplied  with  the 
Benchmark  Suite. 

c.  Automation  of  the  Execution  Benchmarks 

The  compilation  and  implementation  of  the  execution  benchmarks  is  highly  automated  in  the 
Armonics  Benchmark  Suite.  Depending  on  the  computer  system  used,  most  of  the  benchmarking,  from 
installation  to  report  generation,  may  be  accomplished  in  one  or  two  man-days.  In  addition,  once  an 
engineer  has  conformed  the  Benchmark  Suite  to  run  on  a  particular  system,  the  work  can  be  easily 
repeated  as  necessary. 

Compilation  of  the  source  code  of  the  execution  benchmarks  is  explained  in  detail  in  the  next 
section.  The  process  involves  the  use  of  VAX/VMS  contmand  procedures  as  discussed  above  for  the 
compilation  benchmarks.  Once  again,  these  contmand  procedures  may  be  used  directly  on  a  VAX, 
modified  on  systems  which  support  batch  processing,  or  tised  as  a  guide  on  other  systems. 

The  process  of  running  the  benchmarks  is  automated  at  two  levels.  First,  the  benchmarks  them-  « 

selves  (i.e.  the  chosen  CAMP  parts)  are  automatically  executed  by  the  portable  Ada  drivers  in  which  they 
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are  embedded.  Thus,  the  engineer  with  the  task  of  benchmarking  is  not  required  to  supply  an  Ada  driver 
with  which  to  execute  the  benchmarks.  Second,  at  a  higher  level,  the  benchmark  drivers  are  executed 
using  VAX  DCL  command  procedures,  written  for  inclusion  with  the  Benchmark  Suite.  This  level  of 
automation  is,  of  course,  subject  to  system  dependencies. 


» 
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SECTION  V 


USING  THE  BENCHMARKS 

The  following  sections  explain  how  to  perform  benchmarking  using  the  CAMP  Armonics 
Benchmark  Suite.  For  the  purposes  of  discussion,  the  VAX/VMS  environment  is  assumed.  Comments 
throughout  suggest  possible  ways  of  adapting  the  Benchmark  Suite  to  other  environments. 

Table  13  lists  the  DCL  command  procedure  files  which  are  supplied  with  the  Benchmark  Suite.  On 
a  VAX,  these  command  procedures  automate  both  the  compilation  and  the  execution  of  the  benchmark 
drivers.  On  other  systems  the  command  procedures  serve  as  a  guide  although  they  may  be  altered  as 
necessary  to  conform  to  other  batch-processing  systems. 

1.  LOGICAL  DIRECTORIES 

A  system  of  three  directories  is  recommended  for  compiling  and  executing  the  benchmark  code  in  a 
VAX/VMS  environment.  These  directories  are  referred  to  within  the  Benchmark  Suite  command 
procedures  by  the  following  VMS  logical  names: 

•  Compilation_Directory:  The  directory  on  which  all  compilation  takes  place. 

•  Tools:  The  directory  which  contains  the  Benchmark  Suite  command  procedures,  and 

•  Source:  The  directory  which  contains  all  of  the  Ada  source  code  supplied  with  the  benchmarks. 

On  systems  which  do  not  support  the  concept  of  logical  names,  the  command  procedures  may  be 
altered  to  use  the  desired  operating  system  names  and  batch  job  control  style.  Systems  which  do  not 
support  the  concept  of  directories  at  all  may  store  all  of  the  files  (over  360  in  number)  in  a  single  location 
and  alter  the  command  procedures  accordingly. 

2.  USING  THE  COMPILATION  BENCHMARKS 

The  compilation  benchmarks  are  simply  files  of  Ada  source  code.  Testing  a  compiler/linker  system 
with  the  benclunarks  involves  compiling  the  Ada  code  iu  a  correct  order  and  then  linking  the  three 
linkable  main  procedures.  For  VAX/VMS-hosted  Ada  compilers  the  process  is  automatic  depending 
slightly  on  the  command  syntax  used  to  invoke  the  subject  Ada  compiler  and  linker. 

The  file  called  VAX_Compilation_Run.Com  gives  an  example  of  how  to  perform  the  compilation 
and  linkage  editing  of  the  compilation  benclunarks  on  VAX/VMS,  using  the  VAX  Ada  compiler  and  (via 
ACS)  (lie  VMS  linker.  The  procedure  sets  its  process  to  run  in  the  logical  Compilation_Directory  and 
then  calls  the  command  procedure  Compilalion_Benclnnarks  to  perform  the  compilation  and  the  linkage 
editing.  On  other  systems,  the  Conipilation_Bcnchinarks  procedure  gives  a  correct  compilation  order  for 
the  Ada  source  files  and  may  be  used  as  a  guide  or  altered  as  necessary. 


« 
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TABLE  13.  BENCHMARK  SUITE  COMMAND  PROCEDURES 


COMMAND  PROCKDURH 

PURPOSE 

ACI_COMPILATION_RUN 

Call*  Compilation_BeiKhmarks  to  compile/link  the  compilation  benchmark*  on  the  ACT 
compiler. 

ANSI2DV  &  DV2ANSI 

Rename  fife*  from  ANSI  to  development  names  and  reverse. 

CO  MPILATION_BENCH  MARKS 

Compiles/links  source  code  for  the  compilation  benchmarks. 

COMPILEJBENCHMARK.SUPPORT 

Compiles  support  code  for  the  execution  benchmarks. 

COMPILEJTOOLS 

Compiles  the  clock  function  and  I/O  tools  of  the  execution  benchmarks. 

tNT_EXEC  1  _COM_LINK 

Compiles/links  Ada  source  code  for  integrated  execution  1 

INT_EXEC2_COM_LINK 

Compiles/I  inks  Ada  source  code  for  integrated  execution  2 

INT_EXEC.1_COMJ.INK 

Compiles/links  Ada  source  code  for  integrated  execution  3 

MODIFIED_POLYfi_COM_I.INK 

Compiles/links  the  6-digit  precision  polynomial  benchmarks  on  the  TLD  compiler. 

MOD[IIED_POLY9_COM_I.INK 

Compiles/links  the  9-digit  precision  polynomial  benchmarks  on  the  TLD  compiler. 

POLY  fi_COM_LINK 

Compiles/links  the  6-digit  precision  polynomial  benchmarks. 

POI.Y9_COM_l.INK 

Compiles/links  the  9-digit  precision  polynomial  benchmarks. 

SYSTEM_COM_LINK 

Compiles/links  code  to  run  the  System  polynomial  benchmark. 

TLD_BENCHMARKS_COM_LINK 

Calls  other  procedures  to  compile/link  the  benchmarks  on  the  TLD  compiler. 

TLD_COMPILATION_RUN 

Calls  Compi1ation_Benchmarks  to  compile/link  the  compilation  benchmarks  on  the  TLD 
compiler 

VAX_ANALYZE_COM_LINK 

Compiles/links  the  Analyte -Adi  program.  The  program  la  VAX/VMS  and  VAX  Ada 
dependent. 

VAX_  ANALYZE _POI.Y 

Uses  Analyze  Ada  to  analyze  all  of  the  output  from  the  polynomial  benchmarks. 

VAX_BENCHMARKS_COM_LINK 

Calls  other  procedures  to  compile/link  the  benchmarks  on  the  VAX  Ada  compiler. 

VAX_COMPlLATION_RUN 

Calls  Compilation_Benchmarks  to  compile/link  the  compilation  benchmarks  on  the  VAX 
Ads  compiler. 

VAX_INT_EXECI_RUN 

Runs  integrated  execution  1  on  the  VAX. 

VAX_INT_EXEC2_RUN 

Runs  integrated  execution  2  on  tlie  VAX. 

VAX_INT_EXEC3_RUN 

Runs  integrated  execution  3  on  the  VAX. 

VAX_POLY_RUN 

Runs  the  polynomial  benchmarka  on  the  VAX. 

29 


3. COMPILIN';  THE  POLYNOMIAL  AND  INTEGRATED  EXECUTION 
BENCHMARKS 

The  two  classes  of  executable  benchmarks  in  the  Armonics  Benchmark  Suite  must  be  compiled  and 
linked  prior  to  benchmarking.  On  VAX/VMS,  this  process  is  automatic  and  is  accomplished  by  the 
VAX_Benchmarks_Com_Link  command  procedure  provided  with  the  Benchmark  Suite.  This  command 
procedure  establishes  a  process  in  the  logical  Compilation_Directory  and  then  proceeds  to  call  other 
Benchmark  Suite  command  procedures  to  accomplish  the  various  compilation  and  linkage  editing  tasks. 
The  following  command  procedures  are  performed  in  order: 

1. Compile_Benchmark_Support:  compiles  the  CAMP  and  11th  Missile  software  used  in 

benchmarking.  This  software  contains  the  actual  benchmarks  (i.e.,  the  CAMP  parts  selected  as 
benchmarks)  as  well  as  necessary  support  code.  After  compilation,  this  software  comprises  a 
library  of  Ada  units  which  provide  a  context  for  the  subsequent  compilation  of  the  benchmark 
drivers. 

2.  Compile_Tools:  compiles  packages  of  benchmarking  tools  used  by  the  drivers.  These  packages 
are  fully  portable  and  provide  the  drivers  with  necessary  I/O  routines  and  other  utilities. 

3.  VAX_Analyze_Com_Link:  compiles  and  links  the  tool.  Analyze,  used  to  analyze  the  output  of 
the  polynomial  benchmarks.  This  tool  is  dependent  on  VMS  and  VAX  Ada  as  explained  in 
Section  HI.  The  Benchmark  Suite  program  library  dependency  of  this  tool  is  limited  to  the  pack¬ 
age  Benchmarking_Tools.  compiled  by  the  procedure,  Compile_Tools,  just  discussed.  This  means 
that  the  analysis  tool  may  be  independently  compiled  on  VAX/VMS  and  VAX  Ada  and  then  used 
to  check  the  polynomial  benchmark  output  from  other  systems. 

4.  Poly6_Com_Link  and  Poly9_Com_Link:  compile  and  link  the  polynomial  benchmark  drivers. 
Two  command  procedures  are  used:  one  for  the  drivers  using  6-digit  Ada  floating  point  numbers, 
and  one  for  the  drivers  using  9-digit  numbers.  Thus,  Ada  systems  which  do  not  support  the 
extended  floating-point  representations  may  still  compile  the  lower-accuracy  drivers  without  dif¬ 
ficulty.  It  should  be  noted,  however,  that  the  Ada  source  code  files  of  Poly9_Com_Link  will  not 
correctly  compile  unless  those  of  Poly6_Com_Link  have  already  been  compiled.  Two  packages 
necessary  to  the  polynomial  benchmark  drivers  of  both  precisions  are  compiled  in  Poly6_Com_ 
Link. 

5.  Int_Execl_Coni_Link,  lnt_Exec2_Com_Link.  and  lnt_Exec3_Com_Link:  compile  and  link  the 
integrated  execution  benchmarks.  Each  of  these  command  procedures  compiles  the  support  and 
drivers  necessary  to  run  the  respective  integrated  execution  benchmarks. 
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6.  System_Com_Link:  recompiles  CAMP  Polynomials  support  on  the  VAX  then  compiles  and  links 
I  the  System  Driver  bcnclimarks.  This  driver  uses  the  VAX  Ada  math  library  as  a  set  of 

benchmarks.  The  Polynomials  Systein_Functions  LLCSC  interfaces  this  driver  to  the  math 
library.  This  can  be  of  interest  to  users  of  VAX/VMS  and  VAX  Ada  who  must  use  the  VAX  Ada 

*  "slide  rule"  functions,  but  who  have  to  meet  real-time  constraints. 

4.  RUNNING  THE  EXECUTION  BENCHMARKS 
a.  Polynomial  Benchmark  Execution 

Once  compiled  and  linked,  the  polynomial  benchmark  drivers  may  be  run  to  produce  data.  As 
has  been  discussed,  these  drivers  send  output  to  standard  output  which  is  generally  the  console.  On  most 
systems  supporting  file  I/O,  including  VMS,  the  standard  output  can  be  redirected  to  files. 

The  command  procedure  VAX_Poly_Run  is  an  example  of  running  the  polynomial  benchmarks 
in  the  VMS  environment.  Standard  input  is  redefined  to  permit  the  drivers  to  request  their  Input  from  a 
file.  This  file,  created  automatically  by  VAX_Poly_Run  at  benchmark  time,  contains  data  to  supply  the 
benchmark  drivers  with  the  following: 

•  Compiler  Name:  the  name  of  the  compiler  used  to  compile  the  polynomial  drivers  (becomes  part  of 
the  output  data). 

•  Host  Name:  the  name  of  the  compiler  host  machine  (becomes  part  of  the  output  data). 

•  Target  Name:  the  name  of  the  target  machine  of  the  compiler  and  the  machine  on  which  the 
bcnclimarks  will  run  (becomes  part  of  the  output  data). 

•  Number  of  timing  iterations:  the  number  of  times  that  a  driver  must  call  a  function  in  order  to 
achieve  a  certain  accuracy  in  calculating  the  time  for  a  single  call. 

•  Number  of  data  iterations:  the  number  of  data  values  to  use  as  arguments  to  a  function  of  the 
driver.  This  defines  the  number  of  argument-result  pairs  produced  as  output  for  each  benchmark  of 
the  driver. 

Standard  output,  like  standard  input,  is  also  redefined  in  the  case  of  each  benchmark  driver  to 
channel  output  to  files.  This  permits  the  subsequent  analysis  of  the  output  by  the  analysis  program 
Analyze. 

The  analysis  program  is  not  portable  from  the  VMS  and  VAX  Ada  environment  due  to  use  of 
the  VAX  Math_Lib.  Thus,  use  of  the  program  on  other  systems  is  prohibited  unless  modifications  are 
made.  The  program  may,  however,  be  modified  by  interfacing  it  to  another  math  library,  as  long  as  the 
v  output  from  uew  math  library  has  greater  than  nine  Ada  digits  of  precision.  This  is  necessary  since  the 

math  library  is  used  by  the  analysis  program  to  check  the  results  of  the  polynomial  benchmarks,  which 
use  up  to  nine  digits  of  accuracy. 

*  Running  the  analysis  program  is  trivial  and  is  demonstrated  by  the  VAX_Analyze_Poly  com- 
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mind  procedure.  A  user  simply  executes  the  analysis  program,  Analyze,  and  provides  it  with  the  name  of 
a  data  file  produced  by  running  the  polynomial  benchmark  drivers.  The  program  prompts  again  to  re¬ 
quest  the  name  of  the  file  in  which  the  analyzed  output  is  to  be  placed.  After  the  analysis  of  a  file  is 
complete  the  program  starts  over,  requesting  the  name  of  the  next  input  file.  If  no  file  name  is  provided, 
the  program  terminates. 

It  should  be  noted  that,  although  the  analysis  program  is  non-portable,  it  may  be  used  to  analyze 
polynomial  benchmark  data  from  diverse  systems.  A  user  with  access  to  VAX/VMS  and  VAX  Ada  may 
use  the  analysis  program  exclusively  on  that  system  to  check  the  benchmark  output  from  many  other 

systems. 

b.  Integrated  Execution  Benchmarks 

Running  the  integrated  execution  benchmarks  is  similar  to  running  the  polynomial  benchmarks. 
The  command  procedures  VAX_Int_Execl_Run,  VAX_lnt_Exec2_Run,  and  VAX_IntJExec3_Run 
automatically  execute  the  three  integrated  execution  benchmark  groups  on  VMS.  These  procedures 
provide  the  input  data  required  by  the  drivers  while  the  output  of  the  drivers  is  trapped  in  log  files  by 
VMS. 

Like  the  polynomial  benchmark  drivers,  the  integrated  execution  benchmark  driven  use  only 
standard  I/O.  The  input  data  required  by  each  of  the  drivers  is  as  follows: 

•  Compiler  Name:  the  name  of  the  compiler  used  to  compile  the  polynomial  driven  (becomes  part  of 
the  output  data). 

•  Host  Name:  the  name  of  the  compiler  host  machine  (becomes  part  of  the  output  data). 

•  Target  Name:  the  name  of  the  target  machine  of  the  compiler  and  the  machine  on  which  the 
benchmarks  will  run  (becomes  part  of  the  output  data). 

•  Numben  of  Timing  Iterations:  a  series  of  numben  telling  the  driver  how  many  times  to  execute 
corresponding  benchmarks.  Unlike  the  polynomial  benchmarks,  the  different  integrated  execution 
benchmarks  within  a  driver  do  not  all  have  to  be  executed  the  same  number  of  times.  Also, 
overhead  timing  iterations  vary  from  benchmark  to  benchmark.  The  command  procedures  which 
run  the  integrated  execution  benclunarks  on  VAX/VMS  may  be  consulted  for  more  details. 

The  output  generated  by  running  the  integrated  execution  benclunarks  consists  of  two  types  of 

data: 

1 .  Result  data,  which  represents  the  results  of  the  calculations  performed  by  the  subprograms  chosen 
as  benchmarks  and. 

2.  A  table  of  timing  data  showing  the  time  used  for  a  single  call  to  each  benchmark  subprogram. 


Output  data  of  the  first  type  is  to  be  used  in  checking  the  correctness  of  the  data  processing  of  a 
•  tested  system.  Such  output  should  closely  match  the  corresponding  standard  data  supplied  with  the 

Benchmark  Suite.  The  second  type  of  data  represents  the  run-time  efficiency  of  the  tested  system  and  is 
expected  to  vary  widely  from  system  to  system. 

Although  both  kinds  of  data  are  produced  with  each  run  of  an  integrated  execution  benchmark 
driver,  the  correctness  of  the  two  types  is  mutually  exclusive  in  a  given  run.  A  run  which  provides 
accurate  run-time  efficiency  data  is,  by  design,  likely  to  produce  poor  data  for  correctness  checking.  The 
reverse  is  also  true.  For  this  reason,  each  integrated  execution  benchmark  must  be  run  twice,  once  for 
liming  purposes  and  once  to  obtain  data  for  comparison  to  supplied  standard  data. 

When  performing  the  liming  run,  the  number  of  iterations  for  each  benchmark  subprogram 
(specified  by  the  user  at  run-time)  must  be  high  in  order  to  compensate  for  the  generally  low  accuracy  of 
the  clock  functions.  Each  subprogram  will  then  be  called  many  times,  the  lime  for  one  call  being  cal¬ 
culated  by  simple  division.  To  aid  the  user,  each  benchmark  driver  reports  the  number  of  iterations 
necessary  to  obtain  microsecond  accuracy.  Also,  whatever  number  the  user  specifies.  Hie  resultant  table 
of  timings  will  show  estimates  of  the  accuracy  actually  obtained. 

On  the  other  hand,  performing  the  benchmark  run  for  correctness  of  data  processing  requires 
that  the  benchmark  subprograms  be  executed  only  once.  Thus,  the  user  must  specify  that  only  one  itera¬ 
tion  be  used  for  each  subprogram.  More  than  one  call  to  a  given  subprogram  can  alter  the  output  data, 
making  any  comparison  to  the  standard  invalid.  This  is  due  to  the  use  of  in-out  parameters  and  occasional 
side  effects  in  the  benchmark  subprograms.  Results,  in  these  cases,  tend  to  accumulate  changes  from  call 
to  call  as  previously  discussed. 
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APPENDIX  A 


ARMONICS  BENCHMARK  SUITE 

This  appendix  presents  a  summary  of  the  data  which  CAMP  obtained  from  the  Armonics 
Benchmark  Suite,  in  some  cases  the  data  represents  performance  parameters  of  the  selected  CAMP  parts 
as  they  operate  on  a  32-bit  minicomputer.  However,  when  possible,  data  reflecting  the  operation  of  the 
benchmarks  in  a  Mil-Std-1750A  microprocessor  environment  has  been  included. 

The  compilation  benchmark  data  underscores  some  of  the  difficulties  a  software  engineer  may 
experience  when  selecting  or  applying  an  Ada  compiler.  It  was  found  that  many  validated  Ada  compilers 
currently  lack  the  ability  to  handle  complex  source  code.  The  problem  is  essentially  one  of  relative 
reliability:  Some  Ada  compilers  seem  to  work  all  of  the  time;  most  Ada  compilers  seem  to  work  some  of 
the  time. 

The  polynomial  benchmarks,  which  measure  run-time  parameters  of  Polynomials  scientific 
functions,  were  executed  successfully  in  both  the  32-bit  minicomputer  and  1750A  microprocessor  en¬ 
vironments.  This  supplied  us  with  data  enabling  us  to  draw  some  useful  conclusions  about  the  CAMP 
parts,  the  Ada  language,  and  the  tested  compiler/processor  pairs.  Finally,  performance  data  from  the 
integrated  execution  benchmarks  serves  to  validate  these  benchmarks.  At  the  time  of  this  writing  these 
benchmarks  could  not  be  run  in  any  but  the  32-bit  environment  due  to  errors  in  compilation  to  the  1750A 
target  machine. 

I.  COMPILATION  BENCHMARK  DATA 

The  compilation  benchmarks  were  used  to  test  four  separate  Ada  Compiler/Linker  systems.  One 
compiler.  Compiler  A,  was  self-targeted  and  served,  because  of  its  demonstrated  reliability,  as  the  valida¬ 
tion  compiler  for  the  benchmarks.  The  other  Uiree  compilers,  B,  C,  and  D,  were  recently  validated 
cross-compilers  to  a  1750A  target. 

Compiler  A  succeeded  in  compiling  all  of  the  source  code  of  the  compilation  benchmarks.  It 
produced  no  warnings  and  no  errors.  The  accompanying  linker  subsequently  produced  load  modules  with 
no  difficulties.  As  a  final  step,  the  load  modules  were  run  on  the  host  system  to  see  if  they  would  produce 
run-time  errors.  On  this  host,  no  errors  occurred  although  this  implies  no  guarantees  about  other  systems. 

Compiler  B  succeeded  in  compiling  all  of  the  source  code  correctly  except  the  driver  of  the  Kalman 
filter  compilation  benchmark,  Compilation  3.  Numerous  warnings  were  issued  in  the  course  of  compila¬ 
tion.  The  vast  majority  of  these  warnings  concerned  optimizations  which  could  have  been  made  in  the 
Ada  code  but,  for  reasons  of  readability,  were  not.  The  compiler  had  performed  an  optimization  that  was 
not  made  by  the  programmer  at  the  source  code  level.  The  warnings  produced  by  Compiler  B  were 
justified  with  the  exception  of  two  concerning  program  semantics. 

In  compiling  the  Kalman  filter  driver.  Compiler  B  evidently  lost  track  of  a  necessary  file.  Object 
code  was  still  generated  but  it  was  probably  erroneous.  Nevertheless,  all  three  drivers  were  successfully 
linked,  albeit  with  one  warning.  The  linker  of  Compiler  B  produced  the  required  load  modules  and  did 


Dot  fail  to  note  that  Jte  Kalman  filter  driver  had  compiled  with  errors.  The  load  modules  were  next 

loaded  into  the  MDAC-Huntington  Beach  Mil-Std-1 750A  Simulator  and  their  sizes  were  recorded.  < 

Compiler  C  compiled  all  of  the  support  packages  of  the  compilation  benchmarks  but  failed  to  com¬ 
pile  any  of  the  three  drivers.  In  all  three  cases,  the  compiler  ended  abnormally  in  a  late  phase  of  process¬ 
ing.  For  this  reason.  Compiler  C's  linker  could  not  be  fairly  tested. 

Compiler  D  had  been  validated  very  recently  and  appeared  to  be  having  many  of  the  problems 
associated  with  any  new  compiler.  It  failed  to  compile  even  the  support  packages  of  the  compilation 
benchmarks.  After  successfully  compiling  the  first  three  Ada  files,  the  compiler  falsely  diagnosed  the 
fourth  as  having  semantic  errors.  Continuing  through  the  source  code,  the  compiler  found  numerous 
other  "errors"  in  the  error-free  code. 

A  summary  of  the  data  collected  on  Compilers  A,  B,  and  C  is  presented  in  Table  A-l.  Insufficient 
data  was  obtained  from  Compiler  D  to  justify  its  inclusion  in  the  table.  It  should  be  noted  that  the  object 
code  size  data  for  Compiler  A  may  be  unrealistically  small.  The  size  mentioned  in  the  table  does  not 
include  any  run-time  system  services  which  may  be  required. 


TABLE  A-l.  COMPILATION  BENCHMARK  DATA 


COMPILER/ 

SUCCESSFUL 

SUCCESSFUL 

TOTAL  CPU 

TOT  AL  OBJECT 

LINKER 

COMPILE? 

LINK? 

TIME  (nee.) 

CODE SEE 

A 

Ye. 

Ye. 

10:56.56 

62K  byte. 

B 

Most 

Yes 

22:42.33 

122Kbyte. 

C 

No 

NA 

30:00.007 

? 

2.  POLYNOMIAL  BENCHMARK  DATA 

The  polynomial  benchmarks  were  used  to  test  two  subject  systems.  System  A  consisted  of  Compiler 
A,  above,  and  the  host/target  system  of  that  compiler.  System  B  consisted  of  Compiler  B  and  the  MDAC 
Huntington  Beach  Mil-S(d-1750A  simulator,  which  simulates  a  17S0A  bare  machine.  Running  the 
benchmarks  on  System  A  produced  performance  data  on  the  CAMP  Polynomials  package  parts  as  they 
run  on  a  32-bit  time-sharing  minicomputer.  System  B  produced  data  for  the  same  parts  as  they  run  on  a 
20  MHz  1750A  microprocessor.  Compilers  C  and  D,  above,  failed  to  compile  the  polynomial 
benchmarks. 

For  each  function  of  the  Polynomials  package,  size  data  was  obtained  on  System  B.  It  was  felt  that 
1750A  size  data  was  relevant  to  armonics  applications.  Moreover,  this  data  was  readily  available  in  the 
linkage  map  files  produced  by  the  linker  of  System  B.  On  the  other  hand,  size  data  on  the  32-bit  system 
was  less  meaningful  and  was  excluded.  System  A  makes  extensive  use  of  built-in  service  routines  which 
are  not  counted  in  load  size;  on  a  bare  machine,  system  services  arc  part  of  the  load  module  or  run-time 
system  and  are  counted  —  a  fact  which  casts  doubt  on  the  validity  of  code  size  estimates.  Table  A-2  gives 
the  size  data  for  functions  of  the  Polynomials  package  on  System  B. 

Time-consumption  and  mathematical  precision  data  on  the  polynomial  functions  was  collected  for 
both  systems  A  and  B.  This  data  is  summarized  in  Figures  A-l  to  A-12.  Each  graph  plots  the  execution 
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TABLE  A-2.  SYSTEM  B  POLYNOMIALS  SIZES 


TLCSC  Wane 

LLCSC  Name 

Unit  Name 

Size 

Hex. 

{words) 

Dec. 

TLCSC  Name 

LLCSC  Name 

Unit  Name 

Size 

Hex. 

(worda) 

Dec. 

TLCSC  Name 

LLCSC  Name 

Unit  Name 

Site 

Hen. 

{words) 

Dec. 

Chebyshev 

Hod  Newton  Raphson 

Tiylor  Series  (cont.) 

Radian  Operations 

Sqrt 

61 

97 

Degree  Operation! 

Sin  R  5Ter» 

5D 

93 

Newton  Raphson 

Sin  E  5Ter» 

1C 

140 

Degree  Operations 

Sqrt 

6C 

108 

Sin  D  6Term 

4C 

76 

Sin  D  5Term 

5D 

93 

Taylor  Series 

Sin  D  ?Tere 

50 

00 

Semicircle  Operations 

Radian  Operations 

Sin  D  BTerm 

54 

04 

Sin  S  5ter» 

5A 

90 

Sin  R  4Tem 

34 

52 

Cos  D  5Tera 

9D 

157 

Cody  Waite 

Sln~R  5Ter» 

37 

55 

Cos  D  6Tem 

53 

83 

Natural  Log 

Sin  R  6Ter» 

3A 

50 

Cos  D  7Ttrm 

56 

•6 

Mat  Log 

59 

09 

Sin  R  7Tent 

3D 

61 

Cos  D  iTerti 

59 

19 

Base  N 

Sin  R  0Ten» 

40 

64 

Tsn  D  BTerm 

36 

54 

Loq  N 

12 

10 

Cos  R  4Ter» 

4F 

79 

Hod  sin  D  4 Ter* 

76 

ns 

Continued  Fractions 

Cos  R  5Ter» 

52 

82 

Hod  Sin  D  5Tena 

7E 

126 

Radian  Operations 

Cos  R  6Temt 

55 

05 

Hod  Sin  0  6Tera 

•  6 

134 

Tan  R 

31 

49 

Coa  R  7Tent 

58 

88 

Mod  Sin  D  7Term 

IE 

142 

Arctan  R 

36 

54 

Cos  R  0Ter» 

5B 

91 

Mod  Sin  D  0Tem 

96 

150 

Fike 

Tan  R  0Ter» 

2B 

43 

Hod  Coa  D  4Tena 

74 

116 

Semicircle  Operations 

Arcaln  R  5Tem 

22 

34 

Hod  Cos  D  5Term 

7C 

124 

Arcsln  S  4Term 

60 

96 

Arcsln  R  6Tenn 

26 

39 

Hod  Cos  D  6Term 

•  4 

132 

Arcaln  S  5Ter» 

64 

100 

Arcaln  R  7Tem 

2A 

42 

Hod  Cot  D  7Teni 

•c 

140 

Arcsln  S_6Ter» 

es 

104 

Arcsln  R  0Tem 

2E 

46 

Hod  Coa  D  ATera 

94 

148 

Arccos_S  4Ter» 

62 

90 

Arcoa  R  STerm 

29 

41 

Hod  Tan  D  4Tera 

14 

2C 

Arccos  S  5Term 

66 

102 

Arcos  R  6Term 

2D 

45 

Hod  Tsn  D  5Tem 

14 

20 

Arccos  S  6Ter» 

6A 

106 

Arcos  R  7Ter» 

31 

49 

Hod  Tsn  D  6Term 

14 

20 

Hart 

Arcoa  R  0Tent 

35 

53 

Hod  Tan  D  7Tena 

14 

2D 

Radian  derations 

Arctan  R  4Tera 

31 

49 

Hod  Tsn  D  BTerm 

14 

20 

Cos  R  5Ter» 

52 

02 

Arctan  R  5Ter» 

35 

53 

Degree  derations 

Arctan  R  6Ten« 

39 

57 

Cos  D  5T-na 

51 

01 

Arctan  R  7Tera 

3D 

61 

Hastings 

Arctsn  R  0Term 

41 

65 

Radian  Operations 

Alt  Arctan  R  4Ter» 

IE 

30 

Sin  R  4Terra 

36 

54 

Alt  Arctan  R  5Tera 

22 

34 

Sin  R  5Term 

3A 

58 

Alt  Arctan  R  6Tera 

26 

38 

Cos  R  4Teria 

30 

61 

Alt  Arctan  R  7Tera 

2A 

42 

Cos  R  5 Term 

41 

65 

Alt  Arctan  R  STera 

2E 

46 

Tan  R  4Term 

25 

37 

Hod  Sin  R  ?Tera 

6B 

107 

Tan  P  5Ter* 

25 

37 

Hod  Sin  R  5Term 

73 

115 

Arctan  P  6Term 

26 

38 

Hod  Sin  R  6Tera 

7B 

123 

Arctan  R  7Ter» 

2A 

42 

Hod  Sin  R  7Term 

03 

131 

Arctan  R  0Ter» 

2E 

46 

Hod  Sin  R  ITena 

0B 

139 

Hod  Arctan  R  6Ter» 

4C 

76 

Hod  Cos  R  4Ter» 

76 

118 

Hod  Arctan  P  7Ter» 

50 

00 

Hod  Coa  R  5Ter* 

7E 

126 

Hod  Arctan  R  STera 

54 

04 

Hod  Cos  R  6Ter* 

86 

134 

Degree  Operations 

Hod  Coa  R  7Term 

8E 

142 

Sin  0  4  Ter* 

36 

54 

Hod  Cos  R  0Ter« 

96 

150 

Sin  D  5Term 

3A 

58 

Hod  Tan  R  4  Term 

14 

20 

Cos  D  4 Term 

3D 

61 

Hod  Tan  R  5Term 

14 

20 

Cos  D-5Ter» 

41 

65 

Hod  Tan  R  6Term 

14 

20 

Tan  D  4 Term 

23 

35 

Hod  Tan  R  7Ter* 

14 

20 

Tan  I)  5 Term 

23 

35 

Hod  Tan  R  0Term 

14 

20 

time  of  a  function  against  the  absolute  precision  of  that  function’s  results.  Both  the  lime  and  precision 
data  are  taken  over  the  function  argument  domains  listed  at  the  bottom  of  each  figure. 

The  domain  specifications  are  of  particular  importance  since  a  given  function,  apparently  superior  in 
terms  of  performance,  may  nevertheless  operate  correctly  only  over  a  small  domain.  This  is,  for  example, 
true  in  the  case  of  the  radian  arctangent  benchmarks  (Figures  A-6  and  A- 1 2)  where  the  "Alt  Taylor" 
method  appears  to  be  the  best  performer.  However,  referring  to  the  domain  specification,  it  becomes 
apparent  that  the  "Alt  Taylor"  method  only  provides  the  indicated  performance  over  the  domain  (0.0, 0.4|. 
Other  functions  provide  a  more  acceptable  domain  at  a  slightly  higher  throughput  cost. 

The  absence  of  separate  data  for  six  and  nine  digit  instantiations  in  the  figures  based  on  System  B  is 
due  to  the  fact  that  compiler  B  always  uses  1750A  extended  precision  (approximately  9  decimal  digits)  to 
represent  any  generic  floating-point  object.  Identical  object  code  is  used  for  each  instantiation  of  a 
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generic  floating-point  subprogram  and,  indeed,  compiler  B  shares  one  object  code  instruction  section 
amoog  all  instantiations  of  a  given  generic.  The  use  of  this  "single  copy"  method  implies  that  the  running  < 

times  of  different  instantiations  of  the  same  generic  subprogram  will  be  identical,  regardless  of  the  preci¬ 
sion  of  the  floating-point  variables.  Thus,  the  nine-digit  worst  case  data  applies  for  both  six  and  nine¬ 
digit  instantiations. 

3.  INTEGRATED  EXECUTION  BENCHMARK  DATA 

The  integrated  execution  benchmarks,  which  integrate  numerous  CAMP  parts,  were  run  on  the  32- 
bit  minicomputer  (System  A  above.  Tables  A-3,  A-4,  and  A-S). 

Standard  output  data  for  these  benchmarks  is  supplied  in  the  form  of  files  accompanying  the 
benchmark  suite.  This  data  can  be  used  to  verify  that  a  compiler  and  target  machine  combination  produce 
correct  output  for  the  benchmarks.  The  data  is  not  reproduced  here  because  it  is  quite  lengthy  and  is  not 
formatted  for  inclusion  in  a  document. 

Time-consumption  data  on  the  integrated  execution  benchmarks  was  automatically  collected  at  run¬ 
time  by  the  benchmark  drivers.  This  data  is  presented  in  Tables  A-3,  A-4,  and  A-5. 


Number*  shown  Indcat*  lit*  number  of  term*  wed  In 
(he  polynomial  expansion.  The  hxtedon  domain  over 
aNch  thla  data  apple*  I*  |fH,  pl/2). 

Figure  A-l.  Radian  Sine  on  Syslem  A 


160  4 


Minimum  Digits  of  Accuracy 


Number*  shown  Indkata  Via  number  of  term*  used  In 
the  polynomial  expansion.  Tha  function  domain  over 
which  this  data  apples  la  [0,  pi). 

Figure  A-2.  Radian  Cosine  on  System  A 


Numbers  shown  indicate  tha  number  of  term*  used  In 
tha  polynomial  expansion.  The  function  domain  over 
which  this  data  appltae  I*  [-1,  1). 


Numbers  shown  Indeal*  tha  number  of  tarme  tread  In 
tha  polynomial  expansion.  Tha  futctlon  domain*  over 
which  the**  data  apply  are  a*  follow*: 

Taylor  Radian:  [-044,  044) 

Fit*  1  Semicircle:  [-0.0,  0.4] 

Fite  2  Samlclrcla  :  (.14,  1.0) 

Where  FIX  a  1  uses  tha  Newton  Hephoon  square  root  and 
Fit*  2  ueaa  the  Modfffad  Nawton-Raphaon  square  root 


Figure  A-3.  Radian  Tangent  on  System  A  Figure  A-4.  Arcsine  on  System  A 
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34  56789  10 


Minimum  Digits  of  Accuracy 


Numbers  shown  bxdcsls  ths  number  of  terms  used  In 
ttw  polynomlel  sxpenelon.  The  function  domslne  ever 
which  these  dots  eppl y  ere  ee  follows: 

Teytor  Redsn:  [-0.46,  0.48] 

Fbe  1  Semlclrcls:  [4.6,  0.8] 

Fd*  2  Semicircle:  (-IS,  1.0] 

Where  Fdm  1  usee  ths  Newtonflspheon  equera  root  end 
ram  2  usee  tm  ModUsd  NowtorvRophson  equera  root. 

Figure  A-5.  Arccosine  on  System  A 


34  56789  10 


Minimum  Dlglls  of  Accuracy 


34  56789  10 


Minimum  Digits  el  Accuracy 


Numbers  shown  Indlcete  ths  number  of  terms  used  In 
the  potynomlsl  sxpenelon.  Ths  function  domslne  over 
which  disss  dels  eppfy  era  ee  fobows: 

Tsylor:  [2.63,  80.0] 

AR  Tsylor:  (0.0,  0.4] 

Hostings:  (-IjO,  1.0] 

Mod  Hostings:  [0-0,  MS] 

Continued  Fraction:  [0.0,  0.7167] 


Figure  A-6.  Radian  Arctangent  on  System  A 


3456789  10 


Minimum  Digits  of  Accuracy 


4 


Numbers  shown  Indlcsts  His  number  of  terms  used  In  Numbers  shown  Indies  Is  die  number  of  terms  used  bi 

die  polynomlel  oxponslon.  Ths  function  domeln  over  the  polynomlsl  sxpsnelon.  The  function  domeln  over  ( 

which  Mils  dots  oppRee  Is  (-pl/2,  pl/2].  which  this  dels  epplles  Is  [0,  pi]. 

Figure  A-7.  Radian  Sine  on  System  B  Figure  A-8.  Radian  Cosine  on  System  B 

1 


40 


34  56789  10 


Minimum  Digit*  of  Accuracy 


Numbers  thown  Indlcat*  tti*  number  ol  terms  used  In 
the  polynomial  sxpsnalon.  The  function  domain  over 
which  thl*  data  applies  I*  (-1,  1J. 

Figure  A-9.  Radian  Tangent  on  System  B 


3456789  10 


Minimum  Digit*  of  Accuracy 


Numbers  shown  Indlcat*  the  number  of  terms  used  In 
ft*  polynomial  sxpsnalon.  the  funefon  domains  over 
which  the**  data  apply  are  a*  Mow*: 

Taylor  Radian:  1-0.44.  044] 

FIX*  1  Semicircle  (-0.6,  0.4] 

FSn  Z  Semicircle:  (-14,  1.0] 

Where  FIX*  1  uses  tie  Nawton-Raphaon  square  root  and 
FSm  Z  use*  tha  ModHIad  Nawton-Raphaon  square  root 


Figure  A-10.  Arcsine  on  System  B 


34  56789  10 


Minimum  Digit*  of  Accuracy 


Number*  shown  Indlcata  tha  number  ol  term*  used  In 
the  polynomtat  expansion.  Tha  function  domain*  ovar 
which  tha  as  data  apply  are  a*  follow*: 

Taylor  Radian:  (-0.40,  0.44] 

Flka  1  Semicircle:  (-0.6,  0.0] 

Flk*  Z  Samlclrcl*:  (-1.0,  1.0] 

Where  FIX*  1  use*  the  Nawton-Raphaon  square  root  and 
FIX*  Z  uae*  the  MorMed  Nawton-Raphaon  square  root. 

Figure  A- 11.  Arccosine  on  System  B 


34  56789  10 


Minimum  Digit*  of  Accuracy 


Number*  shewn  Indlcata  tha  number  ol  term*  used  In 
tha  polynomial  sxpsnalon.  The  function  domain*  over 
which  these  data  apply  are  a*  follow*: 

Taylor:  (Z.63,  S0.0] 

Alt  Taylor:  (0.0,  0.4] 

Healings:  (-1.0,  1.0] 

Mod  Hasting*:  (0.0,  80.0] 

Continued  Fraction:  (0.0,  0.71(7) 

Figure  A- 12.  Radian  Arctangent  on  System  B 
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TABLE  A-3.  TIMING  OF  INTEGRATED  EXECUTION  1 


Integrated  Execution  1  on  VAX 

TLCSC  Marne  (some  names  abbreviated) 

Time  (microaeca)  1 

LLCSC  Marne 

Unit  Marne 

Per  Call 

Variation 

Waypoint  Steering  (P661 ) 

Compute  Turn  Angle  And  Direction 

391.0 

0 

Compute  Turning  And  Monturning_Dist 

173.0 

0 

Distance  To_Current_Naypoint 

409.0 

0 

Steerinq  Vector  Operations  M  Arcsin 

Initialize 

5210.0 

5 

Update 

2623.0 

10 

Turn  Test  Operations 

Stop  Test 

62.0 

2 

Start_Test 

61.0 

0 

Signal  Processing  (P686) 

Absolute  Limiter 

Limit 

43.0 

0 

Upper  Lower  Limiter 

Update  Limits 

15.0 

0 

Limits 

57.0 

0 

<v 


# 
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TABLE  A-4.  TIMING  OF  INTEGRATED  EXECUTION  2 


Integrated  Execution  2  on  VAX 

TLCSC  Name  (some  names  abbreviated) 

Time  (microsecs) 

LLCSC  Name 

Unit  Name 

Per  Call 

Variation 

Comnon  Navigation_Parts  (P001) 

Update  Velocity 

Reinitialize 

34.0 

2 

Current  Velocity 

51.0 

2 

Update 

Compute  Ground  Velocity 

509.0 

2 

Compute  Gravitational_Accel  Sin_Lat_In 

332.0 

2 

Wander  Azimuth  Navigation  Parts  (P002) 

Earth  Rotation_Rate 

Compute 

338.0 

2 

Earth  Relative  Navigation  Rotat_Rates 
Compute 

414.0 

0 

Total  Platform  Rotation  Rate 

151.0 

0 

Compute  Latitude  Using  Two  Val  Arctan 

623.0 

0 

Compute  Longitude  Usinq  Two  Val  Arctan 

430.0 

0 

Compute  East  Velocity  With  Sin  Cos 

231.0 

2 

Compute  North  Velocity  With  Sin  Cos 

232.0 

2 

Compute  Coriolis  Acceleration 

769.0 

2 

Compute  Wand  Azim  Angle  Two  Val  Arctan 

432.0 

0 

Compute  Curvatures 

820.0 

0 

Direction  Cosine_Matrix  (P644) 

CNE  Operations 

Compute  First  Row  CNE  From  Ortho 

199.0 

0 

CNE  Initialized  From  Reference 

1647.0 

2 

Perform  Rect  Integration  Of  CNE 

664.0 

0 

Reorthonormalize  CNE 

1698.0 

2 

Aligned  CNE  Matrix 

1178.0 

2 

General  Vector_Matrix_Algebra  (P682) 

Matrix  Matrix  Multiply  Restricted 

3861.0 

0 

General_Purpose_Math_Parts  (P 687 ) 

Accumulator 

Accumulate 

19.0 

2 

TABLE  A-5.  TIMING  OF  INTEGRATED  EXECUTION  3 


Integrated  Execution  3  on  VAX 


TLCSC  Name  (some  names  abbreviated) 

LLCSC  Name 

Unit  Name 


Time  (microsecs) 


Per  Call  Variation 


Kalman_Filter_Common_Parts  (P651) 

State_Transition_And_Proc_Noise_Mat_Mgr 

Initialize  1603.0 

Propagate  187290.0 

Get_Current  113.0 

Propagated_Phi  119.0 

Error_Covariance_Matrix_Manager 

Initialize  69.0 

Propagate  149540.0 

P  119.0 

State_Transition_Matrix_Manager 

Initialize  1547.0 

Propagated_Phi  117.0 

Propagate  28073.0 

Kalman_Filter_Compact_H_Parts  (P652) 

Compute_Kalman_Gains  10978 . 0 

Opdate_Error_Covariance_Matrix  16267 . 0 

Opdate_State_Vector  6360.0 

Seq_Opdate_Cov_Matrix  And  State  Vector 

Update  67897.0 

Kalman_Update 

Update  233355.0 

Update_Error_Cov_Matrix_General_Form  73222 . 0 

Kalman_Filter_Complicated_H_Parts  (P653) 

Compute_Kalman_Gains  24687.0 

Update_Error_Covariance_Matr ix  60033 . 0 

Update_State_Vector  12756.0 

Seq_Update_Cov  Matrix_And_State_Vector 

Update  -  192150.0 

Kalman_Update 

Update  361139.0 

Update_Error_Cov_Matrix_General_Form  215098. 0 
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APPENDIX  B 


ADA  SOURCE  CODE  INVENTORY 

The  following  cables  comprise  an  inveniory  of  all  Ada  source  code  used  in  the  CAMP  Armonics 
Benchmark  Suite.  In  addition,  the  tables  provide  a  cross-reference  from  the  development  name  of  a  file 
to  the  ANSI  name  assigned  to  that  file  for  transportation  to  other  operating  systems. 
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TABLE  B-l.  ADA  SOURCE  CODE  INVENTORY 
(1  of  10) 

Amionics  Benchmark  Inventory  and  Cross-Reference 


CAMP  Source  Code 

00  l_000_COM  MON_NA  V_.ADA 

A00 1000.  ADA 

00 1_00 1  _COM  MON_N  A  V.  ADA 

A00 1001.  ADA 

00  l_IOO_ALTm/DE_INTEO  RATION.  ADA 

A00I  I00.ADA 

00  l_200_COMP_GROUND_VEL.ADA 

A00 1200.  ADA 

00  l_300_COMP_GRA  V_ACCEL_LAT_IN.ADA 

A00 1300.  ADA 

00  l_400_COMP_GRAV_ACCEL_SIN_LAT_IN.ADA 

A00 1400.  ADA 

00  l_500_COMP_HEADINO.ADA 

A00 1500.  ADA 

00  l_600_UPDATE_VELOCTTY.  ADA 

A00 1600.  ADA 

00  l_700_SCALAR_VELOCTTY.ADA 

A00I700.ADA 

00  l_800_COM  P_ROT  ATION_INCR.  AD  A 

AOO 1800.  ADA 

002_000_WA_NAV_.ADA 

A002000.ADA 

002_00  l_WA_NAV.ADA 

AOO 2001.  AD  A 

002_  1 00_EAST_VELOCITY.  AD  A 

A002IOO.ADA 

002_200_NORTH_VELOCITY.ADA 

A002200.ADA 

- - - - -  - - - -  — . 

002_300_EARTH_RELJIOR.VELS.ADA 

A002300.ADA 

002_400_TOT AL_  ANOULAR_VEL.AD  A 

A 00 2400.  ADA 

002_500_CORIOLIS_ACCEL.ADA 

A002500.ADA 

002_«00_CORIOLIS_ACCEL_TOT_RATES.ADA 

A002600.ADA 

002_700_RAD_OF_CURV.ADA 

A002700.ADA 

002_800_TOT_PLATFORM_ROT_RATE.ADA 

A002800.ADA 

002_900_EARTH_ROT_RATE  .ADA 

A002900.ADA 

002_A00_EARTH_REL_ROT_RATE.ADA 

A002AOO.ADA 

002_B0O_LATITUDEADA 

AOO2B0O.ADA 

002_C00_LATITUDE_USINO_ATAN.ADA 

A002000.ADA 

002_DOO_LONOITUDE.ADA 

A002D00.ADA 

002_E00_WANDER_ANGLE.ADA 

A002E00.ADA 

002_P00_EAST_VEL_SIN_COS.ADA 

AOO2P0O.ADA 

002_000_NGRTH_  VEL_S  IN_COS.  ADA 

A002000.ADA 

002_H00_EARTH_REL_HOR_VEI.S_SIN_COS.ADA 

AOO2H0O.ADA 

002_l00_LATTrUDE_USINO_AT  AN2.ADA 

A002IOO.ADA 

(K)2_JOO_LONGmJDE_USINO_ATAN2  .ADA 

A002J00.ADA 

002_KOO_W ANDER_ ANOI  E_USlNO_ATAN2.  ADA 

A002K0O.ADA 

61  l_000_WC3S72_MHTRIC_.ADA 

A61 1000.  ADA 

6 1 3_000_WOS72_UNITLESS_.ADA 

A6 1 3000.  ADA 

6  l4_0C0_CONVERS1ON_FACTORS_.ADA 

A6 14000.  ADA 

6 1 5_000_UN  [VERS  AL_CONSTANTS_.  ADA 

A6 15000.  ADA 

62I_000_BDT_.ADA 

A62 1000.  ADA 

TABLE  B-l.  ADA  SOURCE  CODE  INVENTORY  (2  OF  10) 


Almontes  Benchmark  Inventory  and  Cross-Reference 

Development  Name 

ANSI  Name 

62I_00I_BDT.ADA 

A62I00I.ADA 

622_000_KDT_.ADA 

A622000.ADA 

«2_00I_KDT.ADA 

A6 22001.  ADA 

6T4_000_CLOCK_I  IAN  DLER_.  ADA 

A6T4000.ADA 

6T4_001_CLOCK_HANDLER.ADA 

A6T400 1  .ADA 

M4_000_DCM_.ADA 

A 644000.  ADA 

644_00I_DCM.ADA 

A64400I.ADA 

65  l_000_KALM  AN_COM  MON_.  ADA 

A65 1000.  ADA 

MI_OOI_KAt.MAN_COMMON.ADA 

A65IOOI.ADA 

65  l_  I00_PHI_Q_MAN  ACER.  ADA 

A65I  I00.ADA 

65  l_200_P_MANAGER.  ADA 

A65 1200.  ADA 

65  l_300_PHI_M  ANAOER.ADA 

A65 1300.  ADA 

652_000_KAl.MAN_COMPACT_.ADA 

A65 2000.  ADA 

652_OOI_KALMAN_COMPACT.ADA 

A65200I.ADA 

652_IOO_CKO.ADA 

A652IOO.ADA 

652_200_IJPDATE_P.ADA 

A652200.ADA 

652_.TOO_UPDATE_X.ADA 

A652300.ADA 

652_400_UPDATE_P_AND_X-ADA 

A652400.ADA 

652_500_KAl.MAN_tJPDATRADA 

A652500.ADA 

652_600_UPDATE_P_GENERAL.ADA 

A652600.ADA 

65.1_000_KALMAN_COMPLICATED_.ADA 

A65TOOO.ADA 

65T_OOI_KA1.MAN_COMP1.ICATED.ADA 

A65 300 1.ADA 

65T_I00_CKG.ADA 

A653IOO.ADA 

6ST_200_UPDATE_P.ADA 

A65T200.ADA 

65T_.XXI_UPDATE_X.ADA 

A65TT00.ADA 

65.T_400_UPDA  IE_P_AND_X.ADA 

A653400.ADA 

6S.T_5tW_KALMAN_UPDATE.ADA 

A65T500.ADA 

65T_6tn_UPDATE_P_GENERAL.ADA 

A653600.ADA 

66  l_000_WAYPOINT_STEERING_ADA 

A66 1000.  ADA 

66  l_OOI_WAYPOINT_STEE  RING.  ADA 

A66I00I.ADA 

66  l_SOO_STEERINa_VECTOR_OPNS.ADA 

A66 1300.  ADA 

66  |_.T  IO_INmALIZE.  ADA 

A66I3I0.ADA 

66I_T20_UPDATE.ADA 

A66I320.ADA 

66 1  _400_Tl  IRN_  ANGLE.  AND_DIRE(  TtON  .ADA 

A66I400.ADA 

66  l_500_CRSSTRK_AND_l  IDO_ERR_OPNS  .ADA 

A66 1500.  ADA 

66l_5  IO_COMP_WI  [EN_TURNING.ADA 

A66I5I0.ADA 

66l_.520_COMP_WHEN_NOT_nTRNtNO.ADA 

A66 1520.  ADA 

66I_5.TO_COMPUTE.ADA 

A66IJ30.ADA 

66 1_600_DIST_TO_CURR_WA  YPOINT.ADA 

A66 1600.  ADA 

66I_700_COMP_TURN_NQNTURN_DIST.ADA 

A66 1700.  ADA 
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TABLE  B-l.  ADA  SOURCE  CODE  INVENTORY  (3  OF  10) 
Armonicj  Benchmark  Inventory  and  Cross-Reference 


Development  Name 

ANSI  Name 

66I_800_TURN_TEST_OPNS.ADA 

A66 1800.  ADA 

66 1_»  IO_STOP_TEST.ADA 

A  6618  IO.aDA 

66I_820_START_TEST.ADA 

A66 1820.  ADA 

66I_900_STEERING_VECTOR_OPNS_ARCSIN.ADA 

A66I900.ADA 

66 1_AOO_D1ST_TO_CURR_W  AYPOINT.  ARCSIN.ADA 

A661AOOADA 

68  t_000_C_ALCEBRA_.ADA 

A68 1000.  ADA 

68  l_00l_C_  ALGEBRA  ADA 

A68 1001.  ADA 

68  l_200_MATRIX_OPNS  ADA 

A68 1200.  ADA 

68  l_230_SET_TO_IDENTrrY_MATRIX.  ADA 

A68 1230.  ADA 

68  l_240_SET_TO_ZERO_MATRIX  ADA 

A68 1240.  ADA 

68  l_400_MATRDC_SCALAR.OPNS.ADA 

A68 1400.  ADA 

68  l_300_CROSS_PRODUCT  ADA 

A68 1500.  ADA 

68  l_600_MATRIX_VECTOR_MULT.ADA 

A68 1600.  ADA 

68  l_700_MATRIX_MATRtX_MULT.ADA 

A68 1700.  ADA 

682_000_GENERAL_ALGEBRA_.ADA 

A 68 2000.  ADA 

682  00  l_OENERAL_ALGEBRA.ADA 

A68 2001.  ADA 

68 2_  IOO_VECTOR_OPNS_UC.ADA 

A682I0O.ADA 

682_200_MATRDC_OPN  S_UC  ADA 

A682200.ADA 

682_300_DYN.SPARSE_MATRDC_UC.ADA 

A68 2300.  ADA 

682_400_SYMM_HAI.E_STORAOE_MATRDC.ADA 

A682400.ADA 

682_500_SYM  M_FULL_STORAGE_MATRDC_UC.  AD  A 

A68 2500.  ADA 

682_600_D1AGONAL_MATRDCADA 

A682600.ADA 

682_700_VECTOR_SCALAR_OPNS_UC.ADA 

A682700.ADA 

682_800_MATRDC_SCALAR_OPNS_UC.ADA 

A682800.ADA 

682_900_DIAO_MATRDC_SCALAR_OPNS.ADA 

A682900.ADA 

682_A00_MATR1X_MATRDC_MULT_UR.ADA 

A682AOO.ADA 

682_BOO_MATRDC_VECTOR_MULT_UR.ADA 

A682B00-ADA 

682_CW_VECrOR_VECrOR_TRANS_MULT_UR.ADA 

A682COO.ADA 

682_D00_MATRJX_MATRDC_TRANS_MULT_UR.ADA 

A682DOOADA 

682_HOO_DOT_PRODUCT_OPN_UR.ADA 

A682EOO.ADA 

682_POO_D1AO_FULL_MATRDC_ADD_UR.ADA 

A682POO.ADA 

682_QOO_VECTOR_OPNS_C.ADA 

A682000JVDA 

682_llOO_MATRIX_OPNS_C.ADA 

A682H00.ADA 

682J00_DYN.SPARSE_MATRDC_C.ADA 

A682JOO.ADA 

682_KOO_SYMM_EULL_STORAGE_MATRDC.('.AI>A 

A682K00.ADA 

682_IJ)0_VE<TOR_SCALAR_OPNS_C.ADA 

A682UXt.ADA 

682_MOO_MATRDC_SCALAR_OPNS_C.ADA 

A682M00.ADA 

682_NOO_MATRIX_MATRDC_MULT_R.ADA 

A682N00.ADA 

682_POO_MATRD<_  VECTOR.  MUI.T_R, ADA 

A682P00.ADA 

682_QOO_VECTOR_VECTOR.TRANS_MUI.T_R.ADA 

A682Q00.ADA 
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TABLE  B-I.  ADA  SOURCE  CODE  INVENTORY  (4  OF  10) 

Amionics  Benchmark  Inventory  and  Crosa-Reference 


Development  Name 

ANSI  Name 

6B2_R00_MATRIX_MATRIX_TRANS_MULT_R.ADA 

A682R00ADA 

682_SOO_DOT_PRODUCT_OPN_R.ADA 

A682S00.ADA 

6R2_'IW_DIAG_1:ULI._MATRIX_ADD_R.ADA 

A6R2TOO.ADA 

6B2_UOO_VF.CTOR_MATR1X_MULT_UR.ADA 

A682UOO.ADA 

6B2_VOO_VECTOR_MATRlX_Min.T_R.ADA 

A682VOO.ADA 

682_W00_ABA_TRANS_DSP_MATRIX_SQ_MATRIX.ADA 

A682W00.ADA 

682_XOO_ABA_TRANS_VECrOR_SQ_MATRIX.ADA 

A 682X00.  AD  A 

682_YOO_ABA_TRANS_VECTOR_SCALARADA 

A682Y00.ADA 

682_ZOO_COL_MATRIX_OPNS.ADA 

A682Z00.ADA 

683_000_STANDARD_TRIG_.ADA 

A681000.ADA 

683_OOl_STDTRIG_SYSFNS.ADA 

A 68 3001. ADA 

6R4_OOOGEOMETRIC_.ADA 

A684000.ADA 

684_OOI_OEOMETRIC.ADA 

A6S400I.ADA 

l 

6R4_  IOO_UNIT_RAD1AL_VECTOR.AD  A 

A6B4IOO.ADA 

684_200_UNIT_NL_VECTOR.ADA 

A684200.ADA 

6R4_300_SEG_UNlT_NL_VECrOR.ADA 

A684300.ADA 

684_400_OREAT_CIRCLE_ARC_LENaTU.ADA 

A684400.ADA 

6R4_500_SEO_UNrr_NL_VECTOR_ARCSlN.ADA 

A684300.ADA 

686_000_SIGNAL_.ADA 

A686000.ADA 

6«6_001_S10NAL.ADA 

A68600I.ADA 

686_I00_UL_LIMITER.ADA 

A6B6I00.ADA 

6B6_200_U_LIM  ITER  .ADA 

A686200.ADA 

6R6_.100_L_LIM1TER.ADA 

A686300.ADA 

6B6_400_ABS_LIMITER.ADA 

A686400.ADA 

6B6_300_ABS_L1MITER_W_FLAO.ADA 

A686300.ADA 

6R6_600_FIRST_ORDER_FILTER.ADA 

A686600.ADA 

6R6_700_TUSTIN_LAG_FILTER.ADA 

A686700.ADA 

6B6_B00_TUST!N_LEAD_LAa_FILTER  .ADA 

A6B6800.ADA 

6B6_900_SECOND_ORDER_FILTER.ADA 

A686900.ADA 

686_AOO_TUSTIN_INTEGRATOR_W_LIMIT.ADA 

A686A00ADA 

6R6_B00_TUSHNJNT_W_ASYM_LIMtT.ADA 

A686BOOADA 

6B7_000_GP_MAT1I_.ADA 

A687000.ADA 

687_00 1  _GP_MATU.  ADA 

A68700I.ADA 

6B7_IOO_LOOKUP_EVEN.ADA 

A687IOO.ADA 

6B7_200_LOOKUP_UNF.VEN.ADA 

A687200.ADA 

6B7_.100_INCRF.MENTOR.ADA 

A687300.ADA 

6B7_400_DECREMF.NTOR.ADA 

A687400.ADA 

6B7_500_RUN_AVG.ADA 

A687500.ADA 

687_600_ACCUMj\DA 

A687600.ADA 

687_700_CHANGE_ACCUM.ADA 

A687700.ADA 
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TABLE  B-l.  ADA  SOURCE  CODE  INVENTORY  (5  OF  10) 


Aimonks  Benchmark  Inventory  and  Crosn-Refetence 

Development  Name 

ANSI  Name 

687_800_CHANQE_CALC.ADA 

A  68 7800.  ADA 

687_900JNTEORATORADA 

A687900.ADA 

687_AOO_INTERPOIATE.ADA 

A687A00ADA 

687_DOO_RSOS.ADA 

A687D00ADA 

687_BOO_SK3N.ADA 

A687EOO.ADA 

687_P00_MEAN_VAL.ADA 

A687POO.ADA 

(587_000_MAD.ADA 

A687000ADA 

687J*»JjOOKUPJTWOWAY.ADA 

A687HOOADA 

688_00!_POLYNOMIALSADA 

A68800I.ADA 

6*8_200_CHEBYSHEV.ADA 

A688200.ADA 

688_2  !0_RADIAN_OPERATK»NS.ADA 

A6882I0.ADA 

688_220_DEOREE_OPERATIONS.ADA 

A688220.ADA 

688_230_SEM1CIRCLE_OPERATIONSADA 

A688230.ADA 

688_300_PIKEADA 

A688300.ADA 

(58  8_3  lOSEMICIRCLEOPE  RATIONS  ADA 

A6883I0.ADA 

688_400_HART.ADA 

A688400.ADA 

688_4 10_RADIAN_OPERATlONS  .ADA 

A6884I0.ADA 

688_420_DEGREE_OPERATIONS.ADA 

A688420.ADA 

688_500_HASTINGS.ADA 

A688300.ADA 

688_5  IO_RADIAN_OPERATIONS  .ADA 

A6883I0.ADA 

688_520_DEGREE_OPE  RATIONS.  ADA 

A688520.ADA 

688_800_MOD_NEWTON_RAPHSON  ADA 

A688800.ADA 

688_900_NEWTON_RAPHSON.ADA 

A688900.ADA 

688_AOO_TAYL.OR_SERIES.ADA 

A688AOOADA 

(588_AIO_RAD1AN_OPERATIONS.ADA 

A688AI0ADA 

688_A20_DEGREE_OPERATIONS  ADA 

A688A20.ADA 

688_A40_NATURAL_LOG.ADA 

A688A40ADA 

688_A50_BASE_LOG.ADA 

A688A50.ADA 

688_BOO_GENL_POLYNOMIAL.ADA 

A688B00.ADA 

688_O00_SYSTEM_FUNCTIONS.  ADA 

A688COOADA 

688_C  IO_RADIAN_OPNS  .ADA 

A688CIO.ADA 

688_C20_SEMIC[RCLE_OPNS.ADA 

A688C20.ADA 

68  8_C30_DEGREE_OPNS  .ADA 

A688C30.ADA 

688_C40_SQUARE_ROOT.ADA 

A688C40.ADA 

688_C50_BASE_!0.ADA 

A688C30.ADA 

68  8_C60_B  ASE_N .  ADA 

A688C60.ADA 

688_DOO_OONTINUED_FRACT!ONS.ADA 

A688D00.ADA 

688_DIO_RADIAN_OPERATIONS.ADA 

A688DIO.ADA 

688_BOO_CODY_W  AtTE.  AD  A 

A688E00.ADA 

688_E40_NATURAL_LOO.ADA 

A688E40.ADA 
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TABLE  B-l.  ADA  SOURCE  CODE  INVENTORY  (6  OF  10) 


Aimonics  Benchmark  Inventory  nnd  Cross-Reference 

Development  Niime 

ANSI  Name 

688_E50_BASE_N.ADA 

A688E50.ADA 

f>88_POO_REDUCTION.ADA 

A688FOO.ADA 

85  l_000_UNrr_CONVERSION_.  ADA 

A85  IOOO.ADA 

85  i_oo  i_unit_convii  rs  ion  .ada 

A85IOOI.ADA 

890_000_QUATERNION_.ADA 

A890000.ADA 

890_OOI_QUATERNION.ADA 

A89000I.ADA 

890_I00_EULER.ADA 

A890I00.ADA 

890_200_NORMALEED.ADA 

A890200.ADA 

1 1 tli  Missile  Code  (some  modified) 

BARO_ALT_FOR_KF_'tESTS.ADA 

MBAFKFTADA 

BARO_TEST_DRIVER.ADA 

MBROTES.ADA 

DATA_RETRIHVAI._POR_OUlDOPNS_TEST.ADA 

MDT  ARET.  AD  A 

DO_SUM_BARO_AI.TIMOTER_POR_B1AS_TEST.ADA 

MDSUMBA.ADA 

DUMMY_AM.ADA 

MDMMYAM.ADA 

DUMMY_DO_SUM_BARO.ADA 

MDMMYDOADA 

DUMMY_INmALEE_NAVIOATOR.ADA 

MDMMYIN.ADA 

DU  MM  Y_  VELOCTTY.COM  PUT  ATIONS  ADA 

MDMMYVEADA 

EARTH_TO_BODY_TRANSPORM  ADA 

MERTHTO.ADA 

ENVIRONMENT_l:OR_KF_TESTS.ADA 

ME  VIRON  ADA 

EXECUTE_NAVIGATOR.ADA 

MXNAVIO.ADA 

HXECUTE_NAVIGATOR_TESTADA 

MEXECUT.ADA 

EX_NAV_KAI.MAN_Fn.TER_STUB.ADA 

MEXNAVK.ADA 

GU1D_COMPU1ER_FOR_OUIDOPNS_TEST.ADA 

MGUIDCO.ADA 

IN(Y)RPORATI:_KALMAN_CORR.AI  »A 

MINCORP.ADA 

INTERNA1._BUS_BROADCAS1_POR_KP_TESTS.ADA 

M1NTERN.ADA 

ISA_POR_KF_TESTSADA 

MISAPOR.ADA 

KALMAN_Fn.TER_STUB.ADA 

M  KALMAN.  ADA 

M007_  IOO_GUIDANCF._OPNS  ADA 

M007 100.  A  DA 

M007_l  IO_PROCESSOR_MODmED.ADA 

M007I  IO.ADA 

M007_l  1 1_PRINC1PAI._VA1.UE.ADA 

M007III.ADA 

M007_t  t2_PERI=ORM_tNlT.ADA 

M0071 12.  ADA 

M007_t  U_WAYPr_CNTRl._OPNS.ADA 

M007 11.1 .ADA 

M()07_ll4_l1.iam_fONTFOL.ADA 

M0071 14.  ADA 

M007_t  t5_FlRST_ORDER.ADA 

M007I  I5.ADA 

M0 1 2_000_GUIDANCE_DATA_TYPF.S_  .AD  A 

M0I2000.ADA 

M0 1 2_00  I_GUIDANCE_DATA_TYPES.  ADA 

M0I200I.ADA 

M0l.1_000_MISSION_DATA_.ADA 

M0I.1000.ADA 

MO  1 4_000_N  A  V_CO  MPUTER.D  A  T  A  _TYPES_  .ADA 

M0I4000.ADA 

M014_OOI_NAV_COMPUTER_DATA_TYPES.ADA 

MO 14001  ADA 

MO  1 5_00 1  _N  A  V  K3  AT!ON_OPER  ATIONS.  AD  A 

M0I500I  ADA 

TABLE  B-l.  ADA  SOURCE  CODE  INVENTORY  (7  OF  10) 

Armonica  Benchmark  Inventory  and  Croas- Reference 


Development  Name 

ANSI  Name 

MO  1 S_0200_EXECUTE_NA  VKJATORADA 

MO  13020 ADA 

MO  1 5_0900_S  LA  VE_CNE  ADA 

MO  13090 ADA 

MO  1 3_0C00_  B ARO_LOOP_COMPUTAT10NS  ADA 

moi  soar  AD  A 

MOI  5_OHOO_NAV_OPS_TEST_CODEADA 

MO  1  SOHO  ADA 

MOI  7_000_ALKJNMENT_MEASURF.MENTS_.ADA 

MO  17000 ADA 

MO  1  *_000_NA  V_S  YSTEM_.  ADA 

MO 1*000.  ADA 

MO  1 9_000_KALMAN_TYPES_.ADA 

MO  19000 ADA 

MO  I9_00l  _KALMAN_TYPESAD  A 

M0I900I  ADA 

MO  1 9_0 1 00_F_OPERATK5NS .  ADA 

M0I90I0ADA 

MO  1 9_0200_Pf  1 IOPF-RATION  S  .ADA 

MO  19020 ADA 

MOI9_OSOO_ACTIVE_KHPO.ADA 

M0I9080ADA 

MO  1 9_0900_PASSIVE_KHPO.ADA 

MO  19090 ADA 

MO  1 9_OAOO_DOPPLER_KJ  IPO.  ADA 

MOI90AO.ADA 

M02I.000_KALMAN_FILTER_.ADA 

M02 1000 ADA 

M022_000_ENVIRONMENT_.ADA 

M022000ADA 

M024_000_H_ROW_.ADA 

M024000ADA 

M024_OOI_H_ROW.ADA 

M02400I  ADA 

M6 1 1  _000_WOS72_METR1C_.ADA 

M61 1000. ADA 

M6 1 2_000_WGS72_ENGINEERINO_.  ADA 

M6 12000.  ADA 

MEASUREMENTS_FOR_KF_TESTS.  ADA 

M  MEASUR  ADA 

MESSAGE_MANAGER_POR_GUI  DOPNS_TEST.ADA 

MMESSAOADA 

MISSION_DATA_FOR_OUIDOPNS_TEST.ADA 

MMISSIOADA 

NAVIGATtON_OPERATIONS_.ADA 

MNAVIGAADA 

NAV_SYSTEM_STUB.ADA 

MNAVSYSADA 

OCUFORJOLTESTS.ADA 

MOCUPOR.ADA 

SCP_FORJCF_TESTS.ADA 

MSCPPOR.ADA 

TLM_FOR_KF_TESTS.ADA 

MTLMTOR.ADA 

VELOCITY_COMPUTATIONSADA 

MVELOCl.ADA 

VELOCrrY_COMPUT  ATIONS_TEST.  AD  A 

MVELOCT.ADA 

VEL_TEST_DRIVER.ADA 

MVELTESADA 

W  ANDER_ANOLE_COMPUT  ATIONS  .ADA 

MWANDER.ADA 

Compilation  Benchmark  Souice  Code 

IO_WGS72U_.ADA 

CI0WGS7.ADA 

20_NPNAV_.ADA 

C20NPNA.ADA 

2I_NPNAV.ADA 

C2INPNA.ADA 

W_KFOOMMON_.ADA 

C30K1  iCO.  AD  A 

3 1  _KFCOMMON.ADA 

C3IKFCO.ADA 

40_KFCGMPLICATED_.ADA 

C40KFCO.ADA 

4 1  .KFCOMPLICATED.ADA 

C4IKFCO.ADA 

50_POI.Y_.ADA 

C50POLY.ADA 
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TABLE  B-l.  ADA  SOURCE  CODE  INVENTORY  (8  OF  10) 


Amionics  Benchmark  Inventory  and  Cross-Reference 

Development  Name 

ANSI  Name 

5l_POLY.ADA 

C5I  POLY  ADA 

60_OVMA_.ADA 

C60OVMA.ADA 

6l_OVMA.ADA 

C6IOVMAADA 

70_OPMATH_ADA 

C70OPMA.ADA 

7l_OPMATHADA 

C7IOPMA.ADA 

80_CVMA_.ADA 

C80CVMA.ADA 

8  l_CVMA.ADA 

C8ICVMA.ADA 

90_STDTR1G_.ADA 

C90STDTADA 

9I_STDTR10.ADA 

C91STDTADA 

AO_OEO_.ADA 

CAOGEOX  AD  A 

AI_OEO.ADA 

CAIOEOX.ADA 

BO_UNfVCDNST_.ADA 

CBOUNJVADA 

CO_CONVFACTORS_.ADA 

CCOCONV.  ADA 

D0_BDT_.ADA 

CDOBDTX.ADA 

DI_BDT.ADA 

CDIBDTXADA 

E0_WPS_.ADA 

CBOWPSX.ADA 

EI_WPSADA 

CEI  WPSX.ADA 

HO_WOS72_.ADA 

CPOWOS7.ADA 

O0_KDT_ADA 

COOKDTX.ADA 

O  l_KDT.  ADA 

CQ1KDTX.ADA 

Z  l_NP_TDRVR.ADA 

CZINPTD.ADA 

Z2_WPS_TDRVR.ADA 

CZ2WPST.ADA 

Z3JCF_TDRVR.ADA 

CZ3KFTD.ADA 

Original  Benchmark  Source  Code 

68 3A_000_ST  ANDARD_TRIO_.  ADA 

A683AOO.ADA 

683A.OO  l_STDTRiCi_FDCE_HA.STINOS.ADA 

A683AOOADA 

683B_000_STANDARD_TRIO_.ADA 

A683B00ADA 

683B_00  l_STDTRIO_FDCE_HASTINOS  ADA 

A683B00ADA 

68.3_002_STD_TRO_NOSYS.ADA 

A68 3002.  ADA 

687_O0  l_NEWTON_SQRT  ADA 

A687O0I  ADA 

688_000_POL  YNOM  IALS_.ADA 

A688000.ADA 

688_3  IO_SEMICtRa.E_OPE RATIONS  .ADA 

A6883 10.ADA 

ANALYZE.  AD  A 

BANALYZ.ADA 

BENCHMARK  INO_TOOLS.ADA 

BBNMARK.ADA 

BENCHMARKING_TOOLS_.ADA 

BBNCHMA.ADA 

BENaiMARK.CONrENTS.ADA 

BBNCHMR.ADA 

BENCIIMARK_CONTENTS_.ADA 

BBNCHMK.ADA 

CT  1EBYSHEV6_DRIVER.ADA 

BCHEBY6.ADA 

CHEBYSHEV9_DRIVER.ADA 

BCHEBY9.ADA 

CODY6_DRiVER.ADA 

BCDY6DRADA 
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TABLE  B-l.  ADA  SOURCE  CODE  INVENTORY  (9  OF  10) 


Aim  on  ic  ii  Benchmark  Inventory  and  Cross-Reference 

Development  Nome 

ANSI  Name 

CODY9_DRIVER.ADA 

BCDY9DRADA 

CONTINUE  D6.DR1VER.  ADA 

BCNT6DR.ADA 

CONTINUE  D9_DRIVER.  ADA 

BCNT9DR.ADA 

CONTINUED_FRACnON_BENCHMARK.ADA 

BCNTFRAADA 

C0NTINUED_FRACT10N_BENCH  MARK..  ADA 

BCNTFRC.ADA 

CPUCLOCK  .ADA 

BCPUCLOADA 

FKE6_DRIVER.ADA 

BFKE6DADA 

FIKE9_DRJVER.ADA 

BFDCEOD.ADA 

HARTS _DRJVER.ADA 

BHART6DADA 

HART9 .DRIVER. AD  A 

BHART9DADA 

HASTINOS6_DRIVER.ADA 

BHAST6D.ADA 

HASTINOS9_DRTVER.ADA 

BI1AST9D.ADA 

INT_BENCTI  MARK  lNO_TOOLS  ADA 

BINTBEN  ADA 

INT_BENOI  MARK  INO.TOOLS  _ . AD  A 

BINTBNC.ADA 

KALMAN_COMMON_TEST.ADA 

BKALMNC.ADA 

KALM  AN_COMMON_TEST_  ADA 

B KALMAN  .ADA 

KALM  AN_COMPACT_DRIVER.  ADA 

BKLMANC.ADA 

KALM  AN_COMPACT_TEST  .ADA 

BKLMNCO.ADA 

KALMAN_COMPACT_TEST_.ADA 

BKLMCOM.ADA 

KALMAN_COMPLICATED_DRIVERADA 

BKLMNCM.ADA 

KALM  AN_COMPLICATED_TEST  ADA 

B  KJLNCOM  ADA 

KALM  AN_COMPLICATED_TEST_.  ADA 

BKLMCOM.ADA 

M  ATRK.OUTPUT  ADA 

B  MATRIX. AD  A 

MATRDC_OUTPUT_ADA 

BMTRIXOADA 

NEWTON6_DRIVERADA 

BNWTN6DADA 

NEWTON9_DRIVER.ADA 

BNEWTN9ADA 

POLYNOMIALS_NO_SYS_FUNC.ADA 

BPLYNOM  ADA 

POI.YNOMIALS_NO_SYS_FUNC_.ADA 

BPOLYNO.ADA 

POLYNOMLAL.BENCHMARK.ADA 

BPOLYNM.ADA 

POLYNOMIAL_BENOIMARK_.ADA 

BPOLNOMADA 

REDUCE_S  IM_LOG.  AD  A 

B  REDUCE.  AD  A 

SYSTEM_DRIVERADA 

BSYSTEM.ADA 

T A  YLORS.DEGREE  DRTVER.ADA 

BTYLORS.ADA 

TAY1.ORS_RADlAN_DRIVER.ADA 

BTAYLRS.ADA 

TAYLOR9_DEOREE_DRIVER.ADA 

BTYLOR9.ADA 

TAY1.OR0_RADIAN_DRIVER.ADA 

BTAYLR9.ADA 

Benchmark  VAX /VMS  Command  Procedures 

ACT_COMPn.AnON_RUN.COM 

COMPILATION_BENCHMARKS.COM 

COMPtLE_BHNCHMARK_SUPPORT.COM 


JACTCOM.COM 

JCMPD.A.COM 

JCOMPIl-.COM 


TABLE  B-l.  ADA  SOURCE  CODE  INVENTORY  (CONCLUDED) 


Armonics  Benchmark  Inventory  and  Cross-Reference 

Development  Name 

ANSI  Name 

COMPILE_TOOI.S  COM 

JCMPLTO.COM 

INT_EXEC  l_COM_LlNK.COM 

JINTICM.COM 

INT_EXEC2_COM_UNK.OOM 

JINT2CM.COM 

INT_EXF.C3_COM_LINK.COM 

JINT3CM.COM 

MODIFlHD_POLYrt_COM_LINK.COM 

JMDPOL6.COM 

MODIFIED_POLY9_COM_LINK.COM 

JMDPOL9.COM 

POLY6_COM_LINK.COM 

JPLY6CM.COM 

POLY9_COM_LINK.COM 

JPLY9CM.COM 

SYSTEM_COM_LINK.COM 

JSYSCML.COM 

TLD_BENCHMARKS_COM_LINK.COM 

JTLDBCO.COM 

TLD_COMPILATTON_RUN.COM 

JTLDCOM.COM 

VAX_ANALYZE_COM_LINK.COM 

JVAXANL.COM 

VAX_ANALYZE_POLY.COM 

JVAXALY.COM 

VAX_BENCHMARKS_COM_LINK.COM 

JVAXBSC.COM 

VAX_COMPtLATION_RUN.COM 

JVAXCOM.COM 

VAX_1NT_EXEC1_R1  fN.COM 

JVAXI1R.COM 

VAX_IN1_EXEC2_RUN.COM 

JVAXI2R.COM 

V  A  X_INT_EXEC3_RUN.COM 

JVAXI3R.COM 

VAX_POLY_RUN.COM 

JVAXPRU.COM 

ANSI/Development  Name  Conversion 

ANS12DV.COM 

ANSI2DV.COM 

DV2ANSI.COM 

DV2ANSI.COM 

Standard  Output  Data  Files 

VAX_INT_EXEC1_RUN.DAT 

DVXIEIR.DAT 

VAX_INT_EXEC2_RUN.DAT 

DVXIE2R.DAT 

VAX_INT_EXEC3_RUN.DAT 

DVXIE3R.DAT 

IIART6_DRIVER.ANA 

DHART6D.ANA 

HARTrt_DRTVER.DAT 

DHART6D.DAT 

55/56  (Blank) 


INITIAL  DISTRIBUTION  LIST 
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GTE  GOVERNMENT  SYS  CORP 
ADVANCED  DIGITAL  SYSTEMS 
AFATL/FXG 

MILITARY  COMPUTER  SYSTEMS 
LOCKHEED/O/62-81,  B/563,  F15 
HUGHES/FULLERTON 
UNISYS/MS-E 1 D08 
WESTINGHOUSE/BALTIMORE 
AFWAL/AAAS-2 

BOOZ-ALLEN  &  HAMILTON,  INC 
BOEING  AEROSPACE  COMPANY/MS  8H-09 
BOEING  AEROSPACE  CO 
AD/YGE 

SOFTWARE  PRODUCTIVITY  CONSORTIUM 
ARMY  CECOM/AMSEL-COM-IA 
NAVAL  TRAINING  SYS  CENTER/CODE  251 
SCIENCE  APPLICATIONS  INTL  CORP 
J1AYTHEON/MSL  SYS  DIVISION 
CALSPAN 

KAMAN  SCIENCES  CORPORATION 
NAVAL  RESEARCH  LAB/CODE  5595 
CARNEGIE  MELLON  UNIV/SEI/SHOLOM 
COLEMAN  RESEARCH  CORP 
COLSA,  INC 

CONTROL  DATA  CORPORATION 
WINTEC 

CONTROL  DATA/DEPT  1855 
DACS/RADC/COED 
RAYTHEON/ EQPT  DIV 
BMO/AGD 
DDC-I,  INC 

ENGINEERING  &  ECONOMICS  RESEARCH/ 
DIV  OFFICE 
BDM  CORP 
AFATL/FXG/EVERS 
ESD/SYW-JPMO 

FORD  AEROSPACE  &  COMM  CORP/MS  H04 
UN IV  OF  COLORADO  #202 
ANALYTICS 
AFWAL/FIGL 

WESTINGHOUSE  ELECTRIC  CORP/MS  5220 

GENERAL  DYNAMICS/MZ  W2-5530 

HONEYWELL  INC 

TAMSCO 

STARS 

FORD  AEROSPACE/MS  2/206 
GRUMMAN  HOUSTON  CORPORATION 
NAVAL  AVIONICS  CENTER/NAC-825 
NASA  JOHNSON  SPACE  CENTER/EH/GHG 
BOEING  AEROSPACE/MS-8Y97 
HARRIS  CORPORATION/GISD 


1  CARNEGIE  MELLON  UNIV/ 

1  SOFTWARE  ENGINEERING  INST  1 

4  NOAA/ERL/R/E/AL4  1 
1  INTERMETRICS,  INC/G.  RENTH  1 
1  INTERMETRICS,  INC/D. P.  SMITH  1 
1  FORD  AEROSPACE/WEST  DEVEL  DIV  1 
1  AD/ENE  1 
1  ROC  KWELL/MS-GA2 1  1 
1  GRUMMAN  CORP/MS  D-3 1-237  1 
1  INSTITUTE  OF  DEFENSE  ANALYSIS  1 
1  TELEDYNE  BROWN/MS  178  1 
1  USAF/TAWC/SCAM  1 
1  BOEING  AEROSPACE  CO/D.  LINDBERG  1 

5  LOGICON  1 
1  EASTMAN  KODAK/DEPT  47  1 
1  SYSTEMS  CONTROL  TECH,  INC  1 
1  E-SYSTEMS/GARLAND  DIV  1 
1  AFWAL/AAAF  1 
1  MARTIN  DEVELOPMENT  1 
1  MA  COMPUTER  ASSOCIATES  INC  1 
1  IBM  FEDERAL  SYS  DIV/MC  3206C  1 
1  MCDONNELL  DOUGLAS/INCO,  INC  1 
1  UNITED  TECH,  ADVANCED  SYS  1 
1  MCDONNELL  AIRCRAFT  CO/DEPT  300  1 
1  WESTINGHOUSE  ELEC/MS  432  1 
1  MHP  FU-TECH,  INC  1 
1  ITT  AVIONICS  1 
1  COSMIC/UNIV  OF  GA  1 
1  NAVAL  OCEAN  SYS  CENTER/CODE  423  1 
1  NAVAL  WEAPONS  CTR/CODE  3922  1 
1  ODYSSEY  RESEARCH  ASSOCIATES,  INC  1 

USA  ELEC  PROVING  G RD/STEEP  MT-DA  1 
1  PATHFINDER  SYS  1 
1  BDM  CORPORATION  1 
1  PERCEPTRONICS,  INC  1 
1  PHOENIX  INTERNATIONAL  1 
1  MCDONNELL  DOUGLAS  ASTRO  CO  1 
1  GTE  LABORATORY/ RUBEN  PRIETO-DIAZ  1 
1  PROPRIETARY  SOFTWARE  SYSTEMS  1 
1  ADVANCED  TECHNOLOGY  8 
1  STANFORD  TELECOMMUNICATIONS,  INC  1 
1  RATIONAL  1 
1  LOCKHEED  MISSILES  &  SPACE  CO  1 
1  HERCULES  DEFENSE  ELEC  SYS  1 
1  AEROSPACE  CORP  1 
1  ROGERS  ENGINEERING  &  ASSOCIATES  1 
1  ADASOFT  INC  1 
1  ESD/XRSE  1 
1  SANDERS/MER  24-1212  5 
1  CSC/ERIC  SCHACHT  1 
1  COMPUTER  TECH  ASSOCIATES,  INC  1 
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INITIAL  DISTRIBUTION  LIST  (CONCLUDED) 


SCIENCE  APPLICATIONS  INTER  CORP  1 
HQ  CASE/CBRC  1 
GOULD  INC/CSD  1 
HQ  AFSPACECOM/LKWD/STOP  3 2  1 
SVERDRUP/EGLIN  1 
HONEYWELL  INC/CLEARWATER  1 
TECHNOLOGY  SERVICE  CORP  1 
AEROSPACE/LOS  ANGELES  1 
SOFTWARE  ARCHITECTURE  &  ENGIN  1 
LORAL  SYSTEMS  GROUP/D/476-C2E  1 
NADC/CODE  7033  1 
UNISYS/PAOLA  RESEARCH  CTR  1 
SIRIUS  INC  1 
GENERAL  RESEARCH  CORP  1 
SOFTECH,  INC/R.L.  ZALKAN  1 
SOFTECH,  INC/R.B.  QUANRUD  1 
SOFTWARE  CERTIFICATION  INS  1 
SOFTWARE  CONSULTING  SPECIALIST  1 
SOFTWARE  PRODUCTIVITY  SOLUTIONS,  INC  1 
STAR-GLO  INDUSTRIES  INC  1 
NADC/CODE  50C  1 
WESTINGHOUSE/BALTIMORE  1 
MITRE  CORPORATION  1 
SYSCON  CORP/I.  WEBER  1 
SYSCON  CORP/C.  MORSE  1 
SYSCON  CORP/T.  GROBICKI  1 
AEROSPACE  C0RP0RATI0N/M-8-026  1 
TEXTRON  DEFENSE  SYSTEMS  1 
GENERAL  DYNAMICS/MZ  1774  1 
TIBURON  SYSTEMS,  INC  1 
TRW  DEFENSE  SYS  GROUP  1 
NASA  SPACE  STATION  1 
BALLISTIC  MSL  DEF  ADVANCED/ 

TECHNOLOGY  CENTER  1 
IBM  CORPORATION/FSD  1 
VISTA  CONTROLS  CORPORATION  1 
VITRO  CORPORATION  1 
NAVAL  RESEARCH  LABORATORY/CODE  5150  1 
CACI,  INC  1 
AFSC/PLR  5 
DIRECTOR  ADA  JOINT  PROGRAM  OFFICE  1 
MCDONNELL  DOUGLAS  ASTRONAUTICS/ 

E  434/106/2/MS22  7 
SDIO/S/PI  1 
ADVANCED  SOFTWARE  TECH  SPECIALTIES  1 
DTIC-DDAC  2 
AFCSA/SAMI  1 
AUL/L3E  1 


FTD/SDNF 

AFWAL/FIES/SURVIAC 

HQ  USAFE/INATW 

AFATL/CC 

AFATL/CA 

AFATL/DOIL 

6575  SCHOOL  SQUADRON 

IITRI 


1 

1  i 

1 
1 

1  V 

2 

1 

1 


l 
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SUPPLEMENTARY 


INFORMATION 


REPLY  TO 
,  ATTN  OF 


MNOI 


DEPARTMENT  OF  THE  AIR  FORCE 

WRIGHT  LABORATORY  (AFSC) 

EGUN  AIR  FORCE  BASE,  FLORIDA,  32542-5434 


ERRATA.  ^ 


13  Feb  92 


subject:  Removal  of  Distribution  Statement  and  Export-Control  Warning  Notices 


to:  Defense  Technical  Information  Center 
ATTN:  DTIC/HAR  (Mr  William  Bush) 

Bldg  5,  Cameron  Station 
Alexandria,  VA  22304-6145 

1.  Hie  following  technical  reports  have  been  approved  for  public  release  by 
the  local  Public  Affairs  Office  (copy  attached). 


Technical  Report  Number 

AD  Number 

< .  88-18-Vol-4 

ADB 

120  251 

Z.  88-I8-V0I-5 

ADB 

120  252 

S  88-I8-V0I-6 

ADB 

120  253 

-4.  88-25-Vol-l 

ADB 

120  309 

£>.  88-25-Vol-2 

ADB 

120  310 

fc.  88-62-Vol-l 

ADB 

129  568 

n.  88-62-Vol-2 

ADB 

129  569 

88-62-Vol-3 

ADB 

129-570 

9-  85-93-Vol-l 

ADB 

102-654  1 — 

AO.  85-93-Vol-2 

ADB 

102-655 

M.  85-93-Vol-3 

ADB 

102-656 

M.  88-18-Vol-l 

ADB 

120  248 

IS.  88-18-Vol-2 

ADB 

120  249 

\4.  88-18-Vol-7 

ADB 

120  254 

<S.  88-I8-V0I-8 

ADB 

120  255^" 

Mo.  88-18-Vol-9 

ADB 

120  256- 

f7.  88-18-Vol-lO 

ADB 

120  257^ 

1*.88-l8-Vol-ll 

ADB 

120  258 

19.  88-18-Vol-l2 

ADB 

120  259 

2.  If  you  have  any  questions  regarding  this  request  call  me  at  DSN  872-4620. 


LYNNf S. 

Chief,  Scientific  and  Technical 
Information  Branch 


ERRATA 


1  Atch 

AFDTC/PA  Ltr,  dtd  30  Jan  92 


OOTUmKMTOFTHEMIFOftCE 

HEADCKMRTERB  AM  FORCE  OEVEUJFMEWT  TEST  CENTER  <AF8C) 
SQUN  AM  FORCE  BASE,  FLORIDA  32S424000 


REPLY  TO 

attn of:  PA  (Jim  Swinson,  882-3931) 


30  January  1992 


subject:  clearance  for  Public  Release 


TO:  TOj/MNA 


Hie  following  technical  reports  have  been  reviewed  and  are  approved  for 
public  release:  AFAIL-TK-88-18  (Volumes  1  &  2) ,  AFATL-TR-88-18  (Volumes 
4  thru  12) ,  AFAHEL-Tft-88-25  (Volumes  1  &  2) ,  AFATL-TR-88-62  (Volumes  1  thru  3) 
and  AFAQj-TR-85-93  (Volvmes  1  thru  3) . 


N.  PRIBYLA,  Lt  Col, 
Chief  of  Public  Affairs 


AFDTC/PA  92-039 


